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ABSTRACT 

This report describes and evaluates efforts to 
enhance schoox districts' capacity to implement and institutionalize 
the monitoring and management system for an Instructional leadership 
program called Achievement Directed Leadership TADL) . Chapter one 
introduces the report's methodology, limitations, and structure. 
Chapter two first states the purpose of ADL — to help educators use 
research findings to improve student achievement; then it identifies 
crucial variables in achievement and outlines a process for 
influencing these variables. Finally, the chapter describes five 
strategies used to build district capacity: allowing districts 
substantial flexibility in implementation; planning, and orienting 
and training personnel, from the top down; using special activities 
to introduce innovations and build each district's organizational 
skills; providing onsite technical assistance; and directing 
activities toward long-term goals. Chapter three discusses the 
leadership plan for ADL, activities designed to sustain its 
implementation, and support materials* The next two chapters analyze 
the capacity-building process in three school districts. Chapter four 
discusses orientation, planning, and training; chapter five discusses 
implementation, evaluation, and institutionalization. Conclusions 
regarding each of the five strategies and conditions affecting 
implementation complete the report. Appendixes present a summary of 
data collection methods used, tables showing implementation patterns, 
and an interpretation on achievement gains. (MCG) 
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CHAPTER ONE 
INTRODUCTION 

For the past seven years, the Basic Skills Component (BSC) of Research 
for Better Schools, Inc. (RBS) has been developing and testing a research- 
based monitoring and management system to guide educators in the improve- 
ment of elementary school instruction and student achievement in the basic 
skills. _ This Is a special report on the component's efforts to enhance the 
capability of three districts to install the system and sustain its effective 
use. Specifically, this report describes and evaluates strategies BSC and 
the districts employed from April 1981 to June 1984 to foster implementation 
and institutionalization of the system. 

This monitoring and management system, originally designed for class- 
room use, has come to be the central feature of a program of dlstrictwide 
instructional leadership called Achievement Directed Leadership (ADL). The 
program was field tested in three districts in 1981-82. 1 Since then, the 
program has come to include strategies which experience »i< uftented in this 
report) has shown to be effective in promoting Implementation and institu- 
tionalization. 

This chapter describes the data sources and data analysis used in pre- 
paring the report, the report limitations, and the report structure. 



For information on the development of the system see Graeber, A.O. & 
Helms, D.C. (1983).* Documentation Report jkOPhase I; The development of 
Achievement Directed Leadership . Philadelphia: Research for Better 
Schools, Inc. For information on the field test see Blester, T. , Kruse, 
J., Beyer, F. , A Heller, B. (1983). Documentation Report: Phase II: A 
field test of Achievement Directed Leadership . Philadelphia: Research for 
Better Schools, Inc. 



5 



Data Sources and Data Analysis 

The experiences discussed in this report were selected primarily^ from 
BSC's work with three school districts (one each in New Jersey, Pennsyl- 
vania, and Delaware) that cooperated in testing the Monitoring and manage- 
ment system in the 1981-82 school year. This report covers implementation 
in participating elementary schools in those three districts in the field 
test year, 1981-82, and continuance of the program in the following two 
school years, 1982-83 and 1983-84. 

The report is based on three types of data: (1) data on the compo- 
nent *s program development and technical assistance to districts, including 
hypotheses and reflections on the success of program strategies; (2) data 
on district staff activities in preparing for and implementing the program 
and their reflections on those activities; and (3) data on student achieve- 
ment in the basic skills. With the exception of student achievement 
scores, the data presented here, whether supplied by BSC staff or coop- 
erating educators, are mainly observation or self-report data. 

Information on BSC's development of the program and provision of 
technical assistance to the districts was drawn from component documents 
such as staff reports of field work, quarterly reports to the national 
Institute of Education (NIE), special development and evaluation reports 
prepared for NIE, and proposals submitted to the districts. Assessments of 
the success of capacity building and institutionalization strategies were 
solicited from BSC staff during meetings and by memo, verified against data 
on district implementation activities and district staff perceptions, and 
then refined. A similar group "brainstorming" process was used to 



Identify the topics and "learnings" presented in the summary chapter* 
After these topics were agreed upon* individual field staff were asked to 
select confirming vignettes from their field reports or identify relevant 
data from the summary reports of interviews and questionnaires, 

ESf staff used multiple data sources to document implementation 
activities in the district and the district staff T s perceptions of change* 
The major data sources are briefly described below, 

• BSC Contact Reports : Reports were written by BSC staff to 
document all field work. The reports describe objectives, 
activities* outcomes, and future plans for each contact, 

• District Documents : Proposals, plans, correspondence, 
reports, and memos were collected from the districts and 
reviewed • 

• Field Interviews with Superintendents, District Office 
Staff, Principals, and Teachers : Interviews tailored to 
each group were administered. These were generally designed 
to elicit information about roles and activities during the 
year (or previous year), perceived success and problems, and 
overall commitment and reactions to the approach, (See 
Appendix A,) 

• Principal and Teacher Questionnaires : Forms (primarily 
close-ended) directed to each group were designed to obtain 
information on the extent and quality of implementation and 
reactions to the approach. These were administered once 
towards the end of the field test year, (See Appendix A,) 

In order to enhance the reliability of the data* attempts were made to 
cross-check data wherever possible, i,e., the superintendent, central 
office staff, principals, and teachers in each district were questioned 
about the same topics in order to gain insight into their various perspec- 
tives. Consistency of findings among the different sox^cces adds credi- 
bility to the information reported. 

Much of the data on district implementation was qualitative and/or 
self-report, and the analyses were primarily descriptive. Where 
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appropriate, BSC staff quantified interview and questionnaire data in terms 
of frequencies, means, and percentages. The primary level for reporting 
implementation data for this report was the district. Apparent relation- 
ships between implementation data and achievement data were noted, but were 
not subjected to any statistical analyses. 

Each district's existing testing program provided the student achieve- 
ment data for the report. In the three districts, tests were administered 
in the spring of each year to all students. Throughout Delaware, the 
California Achievement Test (CAT) was used in the spring of 1981, 1982, and 
1983; in the spring of 1984, the state switched to the Comprehensive Test 
of Basic Skills (CTBS). The New Jersey school district used the CAT for 
all four years. In addition, scores from the statewide basic skills 
competency test were available for specific grades* In the Pennsylvania • 
district, the Science Research Associates (SRA) achievement test was 
administered in the spring of all four years. In all cases students took 
the level of the test designated by the test publisher as appropriate for 
their grade. 

Normal curve equivalent scores. (NCEs) were used as the basis for 
analysis of student achievement data. Scores were collected for each 
elementary grade in each district (aggregated across schools) in 1981, 
1982, 1983, and 1984. , 

Limitations of the Report 

ADL is a large scale and fairly complex educational innovation. 
Although much can be gained by tracking such an innovation over a three- 
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year pe *d, there were problems «nd constraints associated wi|h this 
documentation effort. The. two major constraints were that: 

• the history and extent of implementation in the three 
districts varied, making it difficult to collect "tmparable 
data from year to year or even In the same year across 
districts 

• some collection and analysis of data that ordinarily would 
have been desirable (e.g., data on the status of individual 
classroom processes related to student achievement gains) 
were impossible given the component's resources and contin- 
uing need to train educators, develop materials, and docu- 
ment activities. Economic constraints precluded the 
collection of hard, objective data, and resulted in much of 

the data being of a self-report nature. 

cs 

This report is not intended to be definitive but rather to share some 
significant BSC experiences and reflections on capacity building. In this 
sense, it is the record of participant observers and, perhaps, subject to 
the same unconscious bias that motivates enthusiastic participants of any 
project. However, this does not mean that thfc data collection was carried 
out haphazardly or that reflection on the experience did not provide 
Indicators of success or failure of Certain BSC strategies. Overall, the 
report is intended to provide a record of experiences and insights that 

others who are working on reseax^h utilization projects may find helpful. 

e 

Structure of the Report 



The rest of this report is divided into five chapters. 

Chapter 2 describes the capacity BSC was trying to build and :he major 
strategies BSC pursued in building that capacity. 

Chapter 3 describes the instructional leadership plan for ADL and 
other processes and materials specifically developed to support implementa- 
tion in the field test districts. 



Chapter 4 describes the Intended and actual orienting, planning, and 
training experiences in three districts. Also discussed are the major 
changes BSC made in Its orientation, graining, and planning procedures 
during the three years, , 

> 

Chapter 5 consists of three aajor sections, each of which is devoted 
to a district. Each section includes:' 

• a brief description of the district and an overview of its 
work with BSC 

• BSC and district strategies used to foster implementation 
along with an assessment of the success of those strategies 

• BSC and district strategies used tor formative evaluation of 
the Implementation 

e the strategies BSC pursued In encouraging the district to 
institutionalize ADL, the strategies or conditions within 
districts that supported or Inhibited institutionalization, 
and an assessment of the extent of institutionalization. 

Chapter 6 'summarizes BSC*s observations on strategies for capacity 

building and institutionalization. 
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CHAPTER TWO 
CAPACITY BUILDING — GOAL AND MAJOR STRATEGIES 

The two major sections of this chapter discuss (1) the capacity BSC 
was attempting to build in each of the districts, and (2) major strategies 
which guided the component in its capacity building efforts. 

The Goal of the Capacity Building Activities 

The component set out to Increase educators' ability to use research 
findings to improve instruction and, ultimately, student basic skills 
achievement. BSC viewed this as a process of helping school districts to 
understand, install, implement, and sustain practices that research suggests 
are conducive to effective instruction. The four major BSC tasks in this 
process were identified as: (1) identify from the research a manageable 
number of variables that educators can influence and that are critical to 
student achievement; (2) develop methods and materials to help educators 
monitor and manage those variables; (3) train educators in the use of 
those methods and materials^ and (4) support educators in their 
implementation of the monitoring and management processes. 

Graeber and Helms (1983) describe the process BSC followed in selecting 
a small set of critical variables and in designing the associated methods 
and materials for tracking and managing them. The variables that were 
identified and the method of monitoring and managing those variables are 
briefly discussed below. Also described is BSC's Intent that districts 
use the variables management system as a method of self-renewal and reform. 

7 



The second half of this chapter presents the major strategies BSC used 
in designing training and technical support for the installation and mainte- 
nance of the monitoring and management system. (The actual training and 
support BSC provided to the districts are described in some detail in 
chapters 4 and 5, respectively.) 

Variables Critical to Student Achievement 

The component's synthesis of research findings on classroom effective- 
ness indicated that students who have, or acquire, knowledge that helps 
them to successfully learn new content, and who spend an adequate amount of 
time covering, mastering, and reviewing content on which they will be 
tested, are much more likely, on year-end achievement tests, to perform 
better than students who do not act this way. 

Consequently, it was inferred that all educators should give special 

attention to the following student behaviors, or variables: 

e prior learning — the knowledge that students have or 

acquire which will help their learning of new subject matter 

• student; engaged time — the amount of time students actually 
spend on assigned learning tasks 

• coverage of criterion content — students* opportunity to 
learn the content on which they will be tested 

• academic performance — sttsdents* success with daily learning 
tasks, their mastery of content units, and their review of 
newly learned subject matter. 

Of course, exclusive attention to one of the behaviors without due 

attention to the others will not be fully beneficial. Furthermore, many 

other variables influence these four behaviors. The following section 

describes how teachers, principals, and central office staff, working 

together, can monitor and manage the four critical student behaviors an1 

also take into account other variables. 

12 
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The Management Process 

The component developed a four-step improvement cycle, or variables 
management process (see Figure 1), to help educators collect data on each 
of the critical variables listed above, and Identify and exploit opportuni- 
ties for improvement. For example, in checking that student engaged time 
Is at an appropriate level, a teacher, with help from the principal or 
another teacher, first collects information on how much time students in 
his or her classroom are actually spending on assigned learning tasks. In 
the second step, this information is compared with similar information from 
research studies. Third, if the comparison Indicates that Improvements are 
needed, the te; ->*r chooses a strategy to increase engaged time. In 
selecting an appropriate strategy educators can also use their knowledge of 
the many other behaviors that influence the target behavior, in this case, 
student engaged time. In the fourth step, the teacher Implements and 
monitors the classroom change. If, after a reasonable time period, the 
change strategy does not show a positive impact on student engaged time, 
the teacher can repeat steps three and four and either adjust the original 
strategy or, if necessary, select a different cne. 



Step I 




Step 3 



Figure 1. Four-step Improvement 



cycle. 
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Capacity for Renewal 

The rather simple Improvement cycle or problem-solving strategy 

i 

portrayed in Figure 1 provides a way of monftoring and managing the 
critical student variables* The component hoped that educators would also 
use the Improvement cycle to monitor and manage student behaviors targeted 
by emerging research findings, as well as the critical leadership functions 
of teachers, principals, and central office staff* In other words, the 
problem-solving strategy was Intended to enhance the district's general 
capability to Identify opportunities for instructional improvement, match 
improvement prescriptions to the opportunities, and monitor and evaluate 

the effects of the modifications. Thus, the capacity-building effort vas 

s 

directed not only toward implementation of a specific Innovation but also, 
toward learning a method of self renewal and reform* In this sense, the 
project and NIE's Research and Development Utilization (RDU) project 
(Louis, Rosenblum, & Molttor, 1981) shared the goal of incorporating a 

problem-solving process into school and district decision-making activities. 

t * 

* Major Strategies in Capacity Building 

When BSC- began its work in 1978, it recognized thatv planned educational 
change had not had a history of broad or consistent success* The component 
hypothesized that by using the findings of research on educational change 
and "inservice education to shape its work, BSC might increase the chanceb 
of success of that work. Therefore, research findings guided the develop- 
ment o£ materials and processes, the component's work with district person- 
nel, and the implementation recommendations and guidelines which BSC provided 
to districts* The five major strategies BSC pursued in its capacity- 
building efforts were; 
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• limit the number of highly specified implementation processes 
and materials in which fidelity of the innovation is Invested 

e orient, plan, and train following a top-down sequence 

e use Innovation-specific Implementation events to help 
districts develop the general planning and organizational 
skills needed to accomplish implementation of the innovation 

e provide on-site technical assistance following training 

e anticipate the probable long-term impact of early design and 
implementation activities. 

The first overall strategy was to limit the number of highly specified 
implementation processes and materials, thereby giving districts flexibility 
in how to accomplish a number of the tasks. BSC hoped that this strategy 

would encourage the input from users which many researchers (e.g., Fullan & 

i 

Pomfret, 1977) claimed was so important for a sense of ownership and for 
successful implementation. 

BSC also adopted, early on, a second strategy: orienting, planning, 
and training in a top-down sequence, i.e., training and decision-making 
were to occur first with district leaders, then central office staff, then 
principals, and then teachers. Despite the notion prevalent in the late 
1970s (which exists to some extent today) that effective improvements spring 
up from grass roots (classroom or building) participation, BSC chose its 
top-down strategy as a means of garnering needed support and leadership 
from the school system bureaucracy. The need for such support had been 
suggested by Brickell (1961) , Barman and McLaughlin (1977), Lipham (1977), 
and Pincus and Williams (1979). The component also selected this model as 
an economical meais of disseminating the approach within the* district. 
Thar is, the district would have, at the central office level, its own 
capacity to train principals and teachers within the district. 
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BSC pursued a third major strategy derived from literature on the 
educational environment's influence on the success of an innovation. 
Pincus and Williams (1979) listed five environmental factors (leadership, 
zone of tolerance, planning and delivery system, derivation of the benefits, 
and stability) that innovators may incorrectly assume to be in place and 
supporting their intended changes. BSC acknowledged the importance of these 
factors and also realized that school districts with the greatest need to 
improve instruction and achlevemep are frequently among the least able to 
provide the support needed to i' element innovations. Some researchers (e.g. , 
Rosenblum & Louis, 1981) have argued thst districts will not change success- 
fully unless their organizational abilities have reached some critical level. 

i ! 

BSC resolved to use actual installation and implementation events to assist 
districts in developing needed organizational behaviors, rather than wait 
for them to develop their organizational abilities before beginning imple- 

9 

mentation. For example, component field agents modeled effective planning 
behaviors by sharing and following a checklist they devised for principals 
to use in planning for teacher training. 

Given favorable circumstances, even with respect to all the environ- 
mental factors, the possibility still remains that the innovation will fail 
if other conditions, such as Intensive inservice for users (Cole, 1971; 

Hamingson, 1973), resource support for change (Berman 4 Pauly, 1975; Downey 

o 

& Associates, 1975), and feedback mechanisms (Charters 4 Pellegrin, 1973; 
Gro.ss, Giacquinta, a Bernstein, 1971) are not met. BSC fqresaw that its 
capacity-building efforts would require a fourth strategyf^provision of 
on-site technical assistance to help assure that those conditions were met. 
BSC planned that the technical assistance would follow the steps of good 
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inservlce listed by Joyce and Showers (1980)™presentation, modeling, 

» 

practice, feedback, and coaching* <\ 

I In the 1970s, the emerging research on educational change clearly 
indicated that program features and implementation decisions influenced the 
extent to which programs would be sustained . As its fifth strategy , BSC 
determined that, whenever possible, all program design and implementation 
tasks would be carried out with a view toward their long-term impact on 
program implementation and institutionalization* For example, Herman and 
McLaughlin (1975) noted that "opportunistic 1 * projects begun in response to 
available funds rarely resulted in lasting or effective change. BSC tried 
to minimize the likelihood that opportunistic districts would implement 
solely because of available funding. Although training was provided free 
of charge, BSC did not provide funds to districts for the teacher inservlce 
time or for defraying other implementation costs. 
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CHAPTER THREE 
PROGRAM DEVELOPMENT TO SUPPORT IMPLEMENTATION 

As the dates for the Initial training of field test district staff 
approached, BSC staff and district leadership grew increasingly concerned 

4 

about how to foster, sustain, and monitor use of the variables management 
process. These concerns prompted BSC to include in the training more 
information on implementation roles and activities and to provide materials 
and specify procedures to support these roles and activities* BSC speci- 
fied the activities which central office staff, principals, and teachers 
would perform as they used the variables management process. BSC also 
specified the activities central office staff and principals need to engage 
in to support principals 1 and teachers' use of the improvement cycle* The 
roles and activities described became known as a "leadership plan," During 
the summer of 1981 the component selected the name Achievement Directed 
Leadership (ADL) for both the leadership plan and the training designed for 
its installation* 

This chapter describes: 

# the overall leadership plan for ADL 

# four major program elements designed to sustain 
implementation of the leadership plan, along with the 
rationale for their inclusion 

# the materials developed to support these implementation 
activities. 

The Leadership Plan 

The component developed the leadership plan to clarify ways central 
office and school staff could coordinate their efforts to establish and 
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maintain instructionally effective classrooms. This plan waa derived from 
research on effective classrooms, schools* and districts. The plan was 
also designed to be compatible with the roles in the traditional school 

i ■ 

district hierarchy^ (superintendents, principals, and teachers). The plan 
prescribes several role-related functions for educators at each level of 
the school district, and describes channels of communication between levels. 
Following the leadership scheme, educators at each level perform their 
functions using the improvement cycle and focusing on the goal of improving, 
or maintaining, levels of the critical classroom variables. The plan also 
provides methods which school and district level administrators can use to 
monitor Implementation of the plan. The following three sections discuss 
how the leadership plan operates at the classroom, school, and district 
levels. 

Classroom 

At the classroom level, the leadership plan (Figure 2) is predicated 

on two related findings: (1) students' classroom behaviors and year-end 

achievement are influenced by behaviors they bring to the classroom at the 

beginning of the year (Bloom, 1976; Cooley & Leinhardt, 1980; Fisher 

et al., 1978); and (2) teachers can, nevertheless, alter conditions of 

teaching and learning and thus influence student classroom behavior and 

year-end achievement (Bloom, 1980; Emmer A Evertson, 1981; Rosenshine, 

1979). Accordingly, the leadership plan calls for teachers to: 

• consider student entering behaviors and use the improvement 
cycle to attend to the critical classroom variables as they 
plan classroom instruction and management 
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• manage the classroom 

• Instruct students * 
— I 

Teaching in the classroom is a complex process >hat occurs in the 
context of larger and even more complex settings, the school and district. 
These more complex settings influence classroom conditions and processes. 
The leadership plan calls for teachers to regularly conference and cooperate 
with their principal to plan and implement improvements. 
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Figure 2. The leadership plan. 



Research has not yet made clear the relationship between specific 
principal leadership tasks and classroon instruction (Koehler, 1981). 
Nevertheless, some research (Wellisch, MacQueen, Carriere, & Duck* 1978), 
and the experience of BSC and its project partner, suggests that principal 
support can help maintain the classroom leadership described in the 
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previous section. Accordingly, the leadership plan for the school level 
calls for principals to; 

• P** 11 program Implementation for the school as a whole- as 
well as plan with individual teachers for classroom 
Improvement 

• train teachers in role-related functions 

• provide teachers with participatory supervision in their use 
of the improvement cycle. ' 

Figure 2 shows a two-way flow of Information between the principal and 
teacher. The principal is continually Informed of classroom conditions 
through regular classroom visits and the review of teachers' instructional 
plans. The principal subsequently uses this information in regular confer- 
ences; with teachers to help them work through the improvement |5rcle. And, 
the leadership plan calls for principals to regularly share this information 
with district level personnel and to plan Improvements with them. 

District 

Research has suggested some characteristics of effective districts and 
critical elements that contribute to a district's success in implementing 
planned change (e.g., Berman « McLaughlin, 1975; Pincus & Williams, 1979). 
However, research does not tell us exactly how these factors affect instruc- 
tional leadership in schools and classrooms, BSC's experience indicates 
that several kinds of central office support can help establish and sustain 
the Instructional leadership described at the school and classroom levels. 

The functions of central office staff are similar to those of princi- 
pals, and are equally concerned with the classroom dimensions which affect 
student achievement. These functions, however, are primarily directed to 

21 
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supporting principals who are expected, In turn, to support teachers. The 
central office functions are to; „, 

• plan with principals 

• train principals to perform their role-related functions 
e provide participatory supervision to principals. 

t: 

As shown in Figure 2, a two-way flow of information should exist 
between central office staff and principals. Although ouch of the comsuni- 
cation will be informal, the principal and district leadership should also 
have formal conferences to review the documented outcomes of the principal's 
conferences with teachers. During these conferences, explicit attention is 
given to the status of classrooms (with respect to the critical student 
variables) and to the teachers' plans for and success in Improving instruc- 
tion. The instructional leadership plan calls for the district leadership 
to be continually alert to opportunities to assist principals with their 
own leadership responsibilities and functions. 

The plan suggests that student year-end achievement be vitally 
Important to the district. District/principal conferences provide an 
opportunity for district leaders to evaluate the classroom information 
compiled by each principal in terms of past student achievement and district 
goals for student achievement in the current year. 

Although the district relies primarily on the principals for informa- 
tion concerning schools and classrooms, district leadership may also 

acquire information directly through personal visits and reports. These 

o 

visits to classrooms are necessarily much less frequent than visits by 
principals. 
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Program El events Designed to Sustain Implementation 

BSC developed the leadership plan described above to provide educators 
at the achool and diatrlct level, with a vision of what they would be doing 
if they were to Implement, support* and sustain use of the improvement 
cycle. However , both BSC staff and district leaders felt a need to provide 
teachers, principals, and central office personnel with specific ways of 
supporting and sustaining use of the variables management procedures. 

BSC had always intended that educators using the Improvement cycle 
participate in activities designed to share data across levels and support 
the implementation. Relatively formal conferences between building princi- 
pals and individual classroom teachers are one example of such an activity. 
Prior to the spring of 1981, however, little attention was given to specify- 
ing these activities or developing materials or training for them. There 
were several reasons for this. The first was ESC's overall strategy which 
suggested that such supporting processes should remain flexible across 
districts and be designed .In coordination with districts so that each 
district would build its ownership of the improvement effort. Furthermore, 
during the early years of program development, BSC staff time was consumed 
by reviewing research and designing training directly related to use of the 
improvement cycle at the classroom level. 

However, by 1981, BSC found that most district and building personnel 
had little time to design methods and materials. BSC also discovered, as 
did NIE's RDU project (Louis et si., 1981), that local development of 
materials could hinder (by claiming scarce lnservice time and attention) 
rather than facilitate improvement efforts. Thus, as training was developed 
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the three year period, it ease to include not only use of the improve- 

< 

cycle at the classroom level, bat also specific implementation act ivi- 
es designed to help educators sustain and support their use of the 
improvement cycle. The training also began to give increased "attention to 
provisions f jt leadership at the building and district levels. The four 
implementation activities that BSC and field test district personnel found 
to be crucial to sustaining implementation were: principal seminars, 
principal supervisory conferences with teachers on classroom data, superin- 
tendent conferences with principals, and differentiated inservice. The 
following discussion describes these foUr elements and clarifies how each 
was intended to help sustain the implementation. BSC's experience with 
district implementation of these elements is discussed in chapter 5. 

Principal Seminars 

Principal seminars were conceived as monthly meetings led by the 
district leadership to coordinate implementation activities, improve 
principals' leadership skills, and keep principals focused on the goal of 
instructional improvement. BSC encouraged the districts to incorporate 
seminars into their usual district/principal meetings, thus communicating 
the expectation that ADL was to be the routine way of operating rather than 
a special project. BSC hypothesized that if district leadership regularly 
showed that instructional improvement was one of their priorities, princi- 
pals would sense its importance and be more likely to establish and main- 
tain instructional improvement as a goal for their school. In seminars, 
the district office staff could address and help principals solve problems, 
whether these were real concerns or only barriers raised to stave off 
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implementation. For example, a common implementation concern of principals 

was lack of time. They claimed that their presetit activities consumed all 

p 

their time and they could not conduct the classroom observations or hold 
teacher conferences. District leadership could explore such concerns with 
principals and resolve the problems in seminars. Seminars could also be a 
source of support to principals who, *in the course of implementation, often 
were applying new skills and behaviors. 

Principal/Teacher Conferences 

The leadership plan suggests that the principal help teachers use the 
improvement cycle and grow in their role-related functions. The principal 
can assist teachers by collecting data (e.g., student engaged time) during 
classroom observations , by encouraging and monitoring teacher collection 
of data on other critical student behaviors (e.g. • academic performance, 
coverage of content) , and by meeting with teachers to review data on all of 
the critical variables. In regularly conducted, formal conferences with a 
teacher, the principal can provide support by recognizing those student 
behaviors that are at levels conducive to good or Improved achievement* 
Together, the principal and teacher can also plan ways of improving those 
student behaviors that are not at satisfactory levels. A principal's 
subsequent modeling of the selected strategies, coaching on the strategies, 
or provision of related inservlce for the teacher are examples of further 
assistance. 

BSC hypothesized that teachers would be prompted to collect data if 
■a number of formal principal /teacher conferences were scheduled throughout 
the school year. However, BSC's reading of the literature suggested that 



25 

22 



ERIC ; 



principal/teacher conferences were often completei usually (with few or no 

» 

provisions for specific conference topics or outcomes) and Made a negligible 

contribution to iaproveaent (Blumberg, ^19^4) , Therefore, BSC developed s 

conference form to provide a basic structure for principal/teacher confer- 

ences. The conference for* called for recording data on the critical * 

variables (prior learning, student engaged time, content coverage, and 

academic performance) along with improvement goals and plans for achieving 

the goals. During training, central office staff and principals practiced 

using the form. The component suggested that principals have at least 

three conferences per year with each teacher, and more frequent conferences 

with teachers who have many opportunities for improvement. 

BSC and the cooperating district leadership realized that the role 

they were proposing for principals might pose problems. The potential 

difficulty is described by Joyce (1982) j 

In most schools there is a tacit understanding between 
administrators and teachers that their respective domains 
are not to be encroached on. Informal sanctions are applied 
to individuals who violate the norms of privacy in the 
classroom, or attempt to generate systematic change within 
the institution that affects working conditions, (p. 48-49) 

Therefore, BSC anticipated that principals would initially resist conduct- 
ing classroom observations and formal teacher conferences. BSC included 
superintendent/principal conferences as an Implementation activity to 
override resistance and encourage principals to proceed with the observa- 
tions and conferences, and to increase t>z district leadership's awareness 
of classroom and school conditions, 

Superintendent/Principal Conferences 

Just as the leadership plan suggests that the role of the principal is 
to support and facilitate the growth of teachers, it suggests that the role 
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of district leaders is to support and promote the growth of principals. 
The district leadership can encourage and monitor principals by Meeting 
with then individually to review the outcomes of their conferences with 
teachers. BSC developed a form principals can use to list the classroom 
data on the critical variables, to identify areas for improvssent, and to 
record agreed upon school building goals and plans. These conferences 
provide the district leadership with an important me is of. assessing the 
principal's skills in planning, training, and supervising teachers, Also, 
the district can use the data on critical classroom conditions and on 
teachers' opportunities for growth to identify appropriate inservice for 
teachers. 

Since the superintendent frequently is the only district office person 
with line authority over principals, BSC hypothesized that It was important 
that the superintendent conduct these conferences. Other central office 
staff not only lack authority over principals, but also often have a tacit 
understanding with building principals about the sanctity of their respec- 
tive domains. 

Differentiated Inservice 

One outcome of the conferences and seminars is the identification of 

m 

improvement opportunities for teachers and principals. When districts 
provide dif fe rent ia ted attention to diagnosed opportunities for improvement, 
they demonstrate use of the variables management process for district 
improvement or reform. Differentiated inservice is not a pre-designed 
element of ADL. However, when such inservice occurs, it shows the district 
is attending to one of ADl/s overall gosls — increased capacity for renewal. 
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BSC developed materials for central office staff and principal training 
that illustrate and suggest how the data collected in conferences and the 
Issues raised in seminars can lead to the development of lnservlce. When 
many principals or teachers share the same diagnosed opportunity for improve- 
ment within or across schools, the district or principal is encouraged to 
deliver group inservice. When opportunities are unique to an individual, 
the district can arrange for the principal's or teacher's participation in 
a specific offering at a local' university or intermediate service agency. 

Support Materials for Implementation Activities 

In the spring of 1981, BSC staff began to develop materials and 
processes related to seminars, conferences, and differentiated inservice. 
^Eor the most part, these materials were incorporated into handbooks — one 
for central office staff, one for principals, and one for teachers. The 
handbooks include information on (1) the leadership plan; (2) roles specific 
to the le^el (e.g., central office staff, principal); (3) principal seminars; 
(A) superintendent/principal and principal/teacher conferences and their 
outcomes; and (5) planning for succeeding years of implementation. These 
materials were explained, and used, during training sessions and seminars. 
The handbooks, which BSC now recommends be distributed and reviewed either 
during the orientation session or the first training session, also include 
the training materials on use of the management process for the critical 
student behaviors. 
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CHAPTER FOUR 

ORIENTING, PLANNING, AND TRAINING 

The earlier chapters of this report described the capacity BSC was 
trying to.bulxd, the major strategies it planned to pursue in building the 
capacity, and the materials and processes developed to assist implementa- 
tion. This chapter describes the component's experience with Installation 
of the innovation in districts. 

The educational change literature includes a variety of multiple stage 
models of the Innovation process. For the purpose of this report, BSC has 
chosen the following six stages: orientation, planning, training, Inplemen 
tation, evaluation, and institutionalization. These stages are not neces- 
sarily sequential or discrete, but are treated separately in this report in 
order to organize the discussion of intents, strategies, and experiences. 
This chapter describes BSC experiences with the first three steps: orien- 
tation, planning, and training. Chapter 5 describes experiences with the 
last three stages. 

Orientation 

Prospective users need to be aware of the nature and purpose of an 
innovation, but awareness is not sufficient to make a decision about 
adoption. The prospective users need to know the challenges they will 
face—particularly where the innovation is at odds with existing norms or 
where it requires new accountabilities. That is, they n need to orient their 
existing system to the innovation to identify points of compatibility and 
stress. 
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BSC's top-down sod el o~ Installation and implementation includes an 
orientation of central office personnel to provide the superintendent and 
others with information needed to help then make an informed decision 
regarding participation. The orientation for central office staff Includes 
information about the amount of inservlce time required, and the implemen- 
tation roles of central office staff, principals, and teachers. 

Orientation was designed to proceed in a top-down fashion so that 
either the necessary approvals and commitments from upper levels *ere 
obtained or the process came to an early halt. If orientation continued 
without such commitment, or began with lower levels, there was a 
possibility that time and resouces would be poorly invested. BSC intended 
that central office staff would orient principals^ and principals, in turn, 
would orient teachers. Orientation sessions for principals and teachers 
were designed to provide information and address questions and concerns. 

The following discussion of experiences, strategies, and outcomes of 
orientation is organized as follows: (1) BSC orientation of the superin- 
tendent and central office staff; (2) orientation of principals; and (3) 
orientation of teachers. Component experiences with orientation in the 
field test districts was limited. Therefore, some experiences outside the 
field test districts that contributed significantly to the component's 
orientation strategies are included in the discussion. 
Orientation of the Superintendent / 

The strategy of first orienting/the superintendent and central office 
staff and seeking their commitment 7 to implementation proved necessary and 

successful in all three field test districts (as well as in other . 

> •■ - / 

/ 

districts) . / 

/ 

/ 

/ 

30 

28 



• Traditionally the New Jersey principals spent the last two 
weeks of August preparing for the opening of school. In 
August 1981, although principals were still negotiating 
their contract, they agreed to spend one of these weeks 
attending AOL training. Without the superintendent's 
committment and insistence on AOL as a priority, it is 
likely the principals would either have made the use of time 
in August a negotiation item or would not have attended. 

• In a New Jersey dissemination district, the central office 
team's attendance and participation in training were excel- 
lent until the superintendent was appointed to a new 
position. At that point, the attendance of other central 
office team members' at training became irregular and 
implementation was not achieved. 

Two factors seemed to influence superintendents' Initial commitment to 

participation and ability to provide resources needed for implementation. 

The first factor, the degree of ^public pressure to improve instruction and 

student achievement, is illustrated by contrasting the Pennsylvania and New 

Jersey districts. The standardized achievement scores in the two districts 

were not radically different. (See Tables 1 and 2 In chapter 5.) However, 

In the New Jersey district there was greater public press for increasing 

achievement. Throughout New Jersey, students in grades 3 and 6 were given 

a state mandated minimum basic skills test; each district's rank on this 

test was published in local newspapers and the scores were the basis of the 

state* a well publicized district classification scheme (approved, approved 

with conditions, or disapproved). In Pennsylvania, participation in the 

state testing program was voluntary, with students in grades 5, 8, and 11 

tested once every three years. The local press paid comparatively less 

attention to test results. Obtaining and holding cousnitment^lri the 

Pennsylvania district was more difficult than in the New Jersey district. 

To hold commitment in the Pennsylvania district, BSC staff found it helpful 

to emphasize that participation supported other district priorities, such 
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as inservice for principals as instructional leaders- BSC also encouraged 
the district to set higher achievement goals, especially for schools where 
achievement was below the district average. 

A second factor, the extent to which the district was loosely coupled 



or had a history of loose coupling, influenced the superintendent's ability 
to mobilize district participation. District leaders varied in their 
capacity to influence changes at the school level. 

• In 1981 the Delaware district was a small, relatively 
tightly coupled district. In this district, BSC's top-down 
orientation, beginning with the superintendent, was a 
successful strategy. 

• In contrast, principals in the Pennsylvania district had 
experienced a masher of years of relative autonomy from the 
central office. Districtwide implementation at the elemen- 
tary level took two years to achieve. Also, the middle 
school principals in the district said they favored the 
program, but they were not persuaded by the superintendent's 
strong encouragement to participate in a district-led 
effort , The middle school principals wanted to decide for 
themselves when and how to go about school improvement. 

Interestingly, the New Jersey district principals also enjoyed con- 
siderable autonomy prior to 1981 (under a series of superintendents with 
short terms), but the superintendent's insistence on full-scale implemen- 
tation, supported by school board press for improvement, seemed to counter 
principal resistance. The superintendent's orchestration of public support 
for increased achievement was a probable key to tightening control. He 
convinced business and community leaders of the need for their help and 
enlisted community support with a BSC-supplied description of the program 
(including a Spanish translation by district staff) in the district's 
newsletter for parents. Clearly, superintendents who wish to tighten 
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district couplings face professional risks. However* there are preparatory 
steps, such as orchestrating private sector support, that can reduce the 
risks. 

Orientation of Principals 

BSC's second major intent for orientation was that central office 

i 

staff orient principals. However, in both the New Jersey and the 
Pennsylvania districts, the central office staff requested and received 
from BSC linkers substantial help in. planning and conducting the 
orientation of principals. (The Delaware district principals had been 
oriented by BSC staff prior to 1981.) Even during the second year (19'82-83) , 
the superintendent and central of Ice personnel in ;.$be Pennsylvania district 
requested BSC assistance with the orientation of 12 principals new to the 
program. However, in all principal orientations, the superintendents gave 
their verbal support to the improvement effort and 9 on their own or at 
BSC's prompting, related the effort to cngoing district initiatives. 

• In the fall of 1981, the superintendent presented a "mission 
statement" to central office staff and principals just prior 
co their initial week of training. The "mission statement" 
Included goals and objectives for the superintendent, 
central office staff, principals, and teachers. The state- 
ment also reflected school board and state policies. The 
mission statement never mentioned Achievement Directed 
Leadership by name, but called for teachers, principals; and 
central office staff to carry out the instructional, leader- 
ship activities that were part of the improvement effort, 

♦ One of the Pennsylvania superintendent's goals was to 
enhance the instructional leadership role of principals. 
During the orientation for principals, the superintendent 
rec00»ende^tlte~proposedT:raIning and implementation as an 
excellent way of helping principals increase their 
leadership role. 
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Generally principals reacted to orientation sessions by agreeing to 
the soundness and desirability of the effort. They also questioned whether 
they would have tine to conduct classroom observations and teacher confer- 



ence*. 



Orientation of Teachers 

The third major intent for orientation, having principals orient 
teachers, was achieved in almost all schools in the New Jersey and Pennsyl- 
vania districts, but not in the Delaware district. BSC provided New Jersey 
and Pennsylvania principals with an outline and overhead transparencies for 
the teacher orientation but, due to inseryice time constraints, only 
minimal training in the use of these materials. Several Pennsylvania 
principals requested central office assistance with teacher orientations. 
In response to a request from the central office, BSC linkers eventually 
provided this assistance. In many cases, district office personnel in both 
New Jersey and Pennsylvania, sensitive to the varied needs and commitment 
of their principals, assigned a team of two principals, or a principal and 
a member of the central office staff, joint responsibility for teacher 
orientation. 

BSC linker visits to teacher orientations in New Jersey and Pennsylvania 
indicated that there was variation in principals' success with the presenta- 
tions. The variation related to principals' skill as trainers and their 
commitment to the program. . 

• In one teacher orientation session observed by a BSC linker, 
the principal followed the orientation outline verbatim, but 
did not give the central office staff person any role in ,fhe 
presentation. The principal did not respond to teacher 
questions and did not call upon teachers who had partici- 
pated in "the development of the effort to share their 
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experiences. The BSC observer felt that the principal's 
style contributed to an atmosphere of distrust and confusion 
among the teachers. 

• At a second school, a principal and central office staff 
person shared the presentation. Although there was some 
Initial ftmbling associated with the sharing, the principal 
reaalned confident and In constant control. Teachers who 
had participated In the development project were invited to 
relate their experiences and add to the presentation. The 
BSC observer described this session as lively and successful. 

In the Delaware district, many teachers were oriented to the program 
prior to 1981. During the field test, the BSC linker provided a brief 
teacher orientation early in the school year. District leaders asked the 
linker to lead the session since it focused on the component's need to 
document implementation for the field test. Central office staff sat in on 
the session but took no part in leading it. 

Although teacher reactions to drlentatlon varied with the quality of 
the presentation, many teachers also expressed concerns about the role of 
classroom observations and student achievement test scores in the teacher 
evaluation process. Teachers' concerns were frequently alleviated during 
the course of the implementation when the practices were put into place. 
Overall , reaction to the orientations was consistent with the report of 
Gall, Haisley, Baker, and Percy (1982), with administrators reacting more 
favorably than teachers to a model that links inservice with the assessment 
of student achievement* 

Conclusions 

There have been no major changes in BSC f s intentions for orientation. 
However, BSC has altered two strategies related to the top-down turnkeying 
of orientations and has developed more materials to ^support orientation 
sessions. 
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BSC's experiences support the need for initial commitment fro* the 
top. However, in loosely coupled districts, initial commitment may be best 
achieved by involving a dec is ion -making team, including principals as well 
as the central office staff, in the initial orientation. If the principals 
reject the innovation and the superintendent has no intention of mandating 
implementation, it is probably prudent for the external linker to accept 
that the district will not participate, rather than continue pressing the 
superintendent for some Implementation. 

Some of the orientation-related materials BSC has developed in the 



with 4 means of assessing their district's readiness to begin training and 

/ 

implementation and with a more detailed understanding of implementation 



equlrements. BSC developed a form designed to help districts compare 



their current goals and practices with improvement effort goals and prac- 
tices. Central office personnel complete the form during their orientation 
and use it to decide whether implementation effort goals are compatible 
with their goals, If they are already implementing significant portions of 
the improvement effort, or if their district lacks conditions or resources 
important to the implementation. Completion of the form not only draws the 
district's attention to the compatibility of improvement effort and district 
goals, but it also serves to aid initial planning for those districts who 
decide to continue with, implementation. BSC has also developed a "Statement 
of Understanding" for use in orienting central office staff. The statement 
lists implementation requirements (such as inservice time) and clarifies 
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RBS and district roles in planning, training, and implementation. If the 
district decides to proceed with planning and training, BSC asks that the 
statement be signed by the district superintendent. • 

A second change in strategy simply Involves providing more time during 
training sessions for central office staff (and principals) to plan for and 
practice the orientation of principals (and teachers). 

Some of the development that BSC has recently pursued Is intended to 
aid external linkers and central office staff in orienting district 
administrators. For example, BSC produced a brief videotape about the 
improvement program and a brochure describing the program and the outcomes 
attained in the field test districts. 

An orientation strategy BSC has considered but never tested is to 
provide <m initial orientation to the superintendent and school board. In 
some districts, BSC has found that the superintendents have oriented their 
own school boards. Since the school board is the body legally responsible 
for the district, explicit school board support for the improvement effort 
might be a persuasive factor in gaining the commitment of principals and 
teachers. 

lanning 

BSC intended that initial planning, like orientation, be carried out 
top-down, beginning at the superintendent's level. Needed planning includes* 
both general logistical concerns (e.g., scheduling inservice time for 
principals and teachers, reproducing materials) and more technical issues 
(e.g., Identifying data available for analyzing students' beginning of the 
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year strengths and weaknesses, and deciding how the district curriculum and 
testing programs be aligned). 

The discussion of experiences and strategies related to planning is 
organized by three major BSC intentions for planning: (1) planning is to 
begin at the superintendent's level; (2) planning for much of the implemen- 
tation for a school year (e.g., 1981-82) should be accomplished early in 
the calendar year (e.g., early 1981); and (3) planning should be ongoing, 
using data generated by implementation of the Improvement program. 
Planning With the Superintendent 

In the three districts, the initial, sustained, and active participa- 
tion of the superintendent in the planning process seemed crucial for 
successful installation and implementation. For\xample: 

a In New Jersey, the superintendent attended almost all of the 
central office/principal training sessions. He initiated 
planning of a timeline of critical events (e.g., teacher 
inservice, observations, conferences, etc.) and saw that 
district administrators adhered to the timeline. Planning 
went smoothly aad implementation in th|e district was timely. 

e In the Pennsylvania district, the superintendent's delegation 
of responsibilities to an administrative assistant resulted 
in serious delays in important decisions (e.g., allocation 
of teacher inservice time). ^- - 

These experiences agree with a conclusion of Rosenblum and Louis 

(1981): "the successful implementation of a districtwlde change program is 

most- effectively facilitated by a chief administrator who dominates both 

the planning process and the administrative decision making in the school 

system" (p. 176). 

Sustaining the superintendent's interest in planning seemed more 
successful when implementation was districtwlde. or at least districtvlde 
at the elementary level. 
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• The Pennsylvania district superintendent ' s attention to 
district planning and delivery of support was such greater 
when all 17 elementary schools were involved, than when only 
5 schools were Involved. 

e In the New Jersey district where all K-8 schools were 

involved <in the fall of 1981, the superintendent was active 
in planning and implementation, could easily direct other 
district resources to the effort (since it involved all 
schools), and could Integrate discussions and training into 
regular meetings with principals. 

• In the Delaware district, the superintendent's involvement 
in planning was not sustained. Only two of the district's 
five school buildings were involved. 

Superintendents often wanted to delegate planning responsibilities 
from the outset. Such delegation usually affected implementation nega- 
tively. For example, delegation, of responsibility in the Pennsylvania 
district resulted in training proceeding without resolution of 
implementation logistics (e.g., provision for inservlce time, specified 
policies on teacher observations and conferences). 

e One Pennsylvania principal interested in implementation 
confirmed this observation: after a r planning session with 
principals (at which no central office staff person was 
present) she took the linker aside and said the other 
principals were not likely to participate seriously in 
planning and training "unless you get [the superintendent] 
to tell them it is a priority and they need to do it. He 
needs to release the [teacher] inservlce time to get this 
going. " 

In the Delaware district, the BSC linker often planned with the 
director of special programs and other district office staff. In this 
instance, the planning seemed effective without the superintendent's direct 
participation, a result perhaps of the district's tight coupling and the 
superintendent's publlcal^r expressed interest in the program. However, 
some aspects of ADL were not implemented as intended (e.g., superintendent/ 
principal conferences), and implementation was not distrlctwide. 
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In the Hew Jersey district, planning was generally easier because: 
(1) the superintendent, himself, readily as suae d responsibility for 
planning: (2) he had purposely United the number of Innovations underway; 
and (3) he set aside time for regular monthly planning sessions with the 
BSC linker. These planning meetings included representatives from the 
principals* and teachers* bargaining units. 
Planning Early in the Calendar Year 

BSC intended that^much planning for the subsequent school year be 
accomplished early in the calendar year. Barriers to this second major 
intent for planning were: (1) insufficient district time for planning; (2) 
loose coupling in the districts; and (3) the absence of conditions or 
practices that the- improvement effort assumed were already in place. 

The decision to conduct a field test was not made until April 1981. 
Thus, district planning for the 1981-82 school year began relatively late 
in the spring and was out of sync with the districts* cycles for budgeting 
and for allocating ^nservice time. In later years, the linkers learned the 
budget and inservice cycles and tried to pace district planning accordingly. 

In the first year, the only viable strategies for furthering implementation 

O' 

were to substitute expedient program methods for more elegant or ideal ones 
and to limit the implementation. 

e In the Pennsylvania district, the late start in planning 
and the numerous reading basal texts used in the five 
schools made it Impossible for dentral office staff to 
complete a text/test match for each series before the school 
year began. There also was no teacher Inservice time to 
plan for year-long Coverage. However, as an expedient, 
teachers and , principals were taught to use descriptions of 
item domains to estimate the extent of coverage at several 
points during the school year. 



40 

38 



• In the New Jersey district, there had been no budgetary or 
logistical planning for acquiring spring test results 
reorganised by fall class rosters. Thus, in planning for 
the year, teachers used the appropriate grade level reports. 

Finding tie* to plan ahead with central office personnel (including 
but not United to the superintendent) was difficult. 

e The Pennsylvania district had a number of projects underway. 
^ Central office staff were often diverted jtn so many direc- 
tions that scheduling planning sessions was difficult. The 
superintendent acknowledged this situation but felt strongly 
that by providing principals with a wide array of projects, 
each was no re apt to find one. that would work for him or 
her. This approach- taxed the central office support system 
and placed a heavy planning burden on principals. 

e In the New Jersey district, the superintendent reviewed 
existing projects and eliminated some to reduce the number 
of demands on central office staff and principals. By doing 
ao, central office staff were free to work on a major 
curriculum documentation effort during the summer of 1981. 
That effort was central to the content component of the 
$ improvement effort. 

Strategies that linkers adopted to obtain planning time and to make 
the best possible use of that time included: 

e Identifying what decisions were needed and scheduling for 
year. 

• Establi*»«xng a regular, monthly planning session with the 
superintendent. 



e Listing decisions in need of immediate attention (along with 
some alternatives based on the linker's knowledge of the 
district) prior to planning sessions with the superintendent. 
Advanced planning by the linker with other involved central 
office staff aided the process. 

• Learning the district budgeting and planning cycles so that 
decisions required for implementation could be included for 
consideration and support. 

• Writing a monthly briefing letter to the superintendent 
reviewing recent decisions and accomplishments, as well as 
impending decisions. This alerts the superintendent to 
planning needs and to issues requiring his attention. 
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Another substantial hurdle in the planning process mis the relative 
independence, or loose coupling, of district personnel responsible for the 
curriculum, testing, and allocation of inservicc time. Since the improve- 
* metft approach involves the alignment of curriculum and testing, the use of 
test data to analyze prior learning, and the use of inservlce time to 

respond to identified opportunities for improvement, coordination among the 

* 

three areas is crucial to successful implementation. Inclusion of relevant 
staff in the orientation and training sessions did not assure their later 
participation or cooperation* In all three test districts, the Superinten- 
dent was the only person who had the authority to require communication or 
cooperative planning. Educating the superintendent about the necessity of 
such cooperation and motivating the superintendent to tighten the couplings 
proved to be substantial tasks but were the only BSC strategies that held 
much promise of success* 

BSC staff also spent considerable time helping central office staff 
plan for tasks • some of which BSC assumed districts would have completed 
prior to BSC involvement. For example, it was assumed that districts would 
have aligned their testing program with their district curriculum (or basal 
text series). Similarly, two of the three districts had no requirements 
for mastery testing or criteria for judging mastery. Overall, a substantial 
amount of BSC technical assistance was directed toward helping districts 
put Into practice conditions prerequisite to Implementation. 

Continue Planning Using Data 

BSC f s third intention for planning was that district leaders use data 
to assess progress and to plan additional training or resources, throughout 
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the year. However, districts were not accustomed to using formative 
evaluation data for decision making. First, central office staff and 
principals tended to be sidetracked by peripheral benefits, and second, 
central office staff and principals were accustomed to the expediency of 
using informal testimony from selected individuals as a basis of decision 
staking. 

• In the third year of Implementation In the Hew Jersey 
district, the superintendent was so convinced that the 
principals were carrying out their roles as instructional 
leaders, and so involved in sharing his district's progress 
with audiences outside the district, that his attention to 
principals declined. He relied on principals' assurances 
that things were progressing well rather than data available 
to him. End-of-year Interviews indicated that implementa- 
tion had declined and test results in the spring of 1984 
also showed a slight drop. 

• The reading/language arts specialist in the Pennsylvania 
district used reports of certain teachers to reinforce her 
claim that all teachers viewed the effort as one of teaching 
to the test. End-of-year interviews with a sample of 
teachers from each school indicated that this view, while 
evident, was not pervasive. 

The strategy that linkers most frequently pursued to encourage the district 
use of data for formative evaluation was to model the behavior, that is, 
show by example how data that had b*en collected could be used in planning. 
The reaction to such modeling was generally quite positive. 

• In 1983, the BSC linker met with the New Jersey district 
reading/ language arts and mathematics supervisors to design 
an analysis of districtwide test results that would suggest 
areas where the curriculum sequence or the instructional 
materials might be weak. Although the supervisors had pre- 
viously used test data to rank schools and to identify low 
achieving classes, they had not used the data to diagnose 
weak program areas. They were enthusiastic about the 
process once they were shown how it was done, and they 
repeated the analysis in 1984. 
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Conclusion 
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Although BSC's intentions for planning have not changed during the 
three-yeir period, three changes were made in strategies related to 
planning. First, the component Increased the amount of information about 
planning provided to central office staff during orientation sessions. 
Helping districts through the process of comparing their present practices 
with those of the improvement system increased the districts' awareness of 
the amount and type of planning that is needed to improve instructional 
effectiveness. Second, the component built major planning sessions into 
the timeline of implementation events presented during the orientation. 
And third, planning guides were written for some of the tasks BSC had 
erroneously assumed either had been completed by districts (e.g., aligning 
curriculum and testing), or were already part of districts' routines (e.g., 
completing logistical arrangements for teacher training). These planning 
guides were incorporated into BSC training materials. 

Training 

The training originally designed by BSC was intended to provide each 
level of the district (central office, principals, teachers) with knowledge 
of the critical classroom variables and the variables management process. 
In addition, the training for central office staff and for principals was 
intended to help them become trainers of principals and teachers, respec- 
tively. The training for each level was divided into two major segments, 
one on managing instructional time and one on managing instructional 
content. The training intended for central office staff and for principals 
was similar; the training for teachers was considerably shorter as fewer 
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t skills were Included. Although each of the two segments included some ' 
■r • . 
attention to the related tasks and responsibilities at each level, this was 

15 

not a major emphasis of the training as delivered in the fall of 1981. 

As with orientation, it was Intended that once BSC had trained central 
office staff, they would then train principals who, in turn, would train • 
teachers. The training for central office staff and principals included 
some attention to their training of principals and teachers. However, 
given the press of time, this attention was limited. 

The discussion of training experiences and strategies begins with some 
observations on the difficulty of scheduling training, and is then structured 
by the BSC intentions that (1) BSC train central office staff who in turn 
train principals, and (2) principals train teachers. 

Scheduling Training 

Scheduling training proved difficult as the districts were not accus- 
tomed to devoting the amount of time requested for initial lnservice (25 
hours for central office staff, 22 hours for principals, 6-9 hours for 
teachers) to one effort. District calenders included sufficient lnservlcj 
time over the school year, but it was typically devoted to a number of 
different projects or topics. 

Superintendents were very concious of how they spent their time. They 
argued that the present training for central office staff was too lengthy. 
While it was essential that they understand the ADL processes, It was not 
essential that they become highly skilled in every aspect of the training 
(e.g., classroom observation). In the future, BSC may tailor training more 
specifically to superintendents. The component is contemplating training 
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superintendents in a brief academy and suggesting that such an academy be a 
prerequisite to training other district personnel. 

Two strategies BSC used in the field test districts to deal with the 
lack of inservice tine for all district administrators were to shorten the 
time spent on initial training and to spread training sessions out over the 
course of the school year. 'However, there were problems Inherent in both 
of these strategies. When inservice for central office staff and principal 
was conducted within one week (as was the case in the Mew Jersey district 
and other non-field test districts), participants reported that the week of 
training was too intense. BSC usually had to provide additional training 
tc principals during the year to clarify ideas and correct misunderstand- 
ings; time originally set aside to address principals' implementation 
concerns was used for training. The second strategy, spreading training 
over the year, meant that implementation of some aspects of ADL was delayed 
Also, some staff involved in this long-term training reported losing sight, 
of the entire process. 

Obtaining sufficient inservice time for training teachers was often 
even more difficult. In some districts, different levels of the system 
seemed to have informal "rights" to teacher inservice time. For example, 
in the New Jersey and Pennsylvania districts, the central office staff 
determined how some inservice hours would be spent and building principals 
were responsible for allocating the remaining inservice hours. An addi- 
tional problem in all three field test districts ''18 that much of the 
1981-82 teacher inservice time had been assigned to other topics prior to 
the commitment to implement the improvement effort. 
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• In the Pennsylvania district, all 1981-82 teacher inservice 
tine had been assigned prior to June 1981. Nevertheless, 
given the superintendent * s concern, the five principals 
found ways to wake the tine available for teacher inservice. 
In that district In the 1983-84 school year, the district 
vas able to allocate sufficient teacher inservice tine for 
teacher training in all elementary schools. 

a In the Rev Jersey district, the inservice time specified in 
the teachers' contract was less than that needed for the 
improvement effort. However, the superintendent authorised 
use of two additional half school days for teacher Inservice 

Thus, while teacher inservice time vas considered a scarce commodity 

and had been planned for or assigned in advance, the Pennsylvania and New 

Jersey superintendent were able to adjust calendars to address their 

priority. At times, the Inservice time secured was somewhat less than that 

requested, but overall substantial amounts of time were made available. 



Training Central Office Staff and Principals 

One intention concerning how training was to be conducted was that the 
BSC train central office staff who in turn were to train principals. 
However, in none of the field test districts did central office staff, on 
their own, conduct training for principals. While the relatively short 
period of time available for preparation in the field test districts (April 
to September) may account for central office staff not training principals, 
BSC experience In other districts suggests that tradition and organizational 
concerns were also barriers. In the three field test districts, central 
office staff had previously provided inservice workshops directly to 
teachers, but not to principals. The line of authority in districts also 
seems to be a major factor. For example, in the Pennsylvania district, 
principals reported directly to the superintendent, and linkers sensed that 
other central office "staff were reluctant to play any role that might even 
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suggest authority over principals* A number of central office staff nave 
Indicated that their training of principals was less effective than training 
supplied by external linkers. When they do train principals they are shown 
as little regard as the proverbial prophets in their own land. 

In the field test year, principals and central office staff in each of 

the three districts were trained simultaneously by the BSC linker. 

.. v 

e In the Delaware and. New Jersey districts, the superintendent's 
and assistant superintendent's attendance at these sessions 
enabled them to stake decisions about implementation Issues 
(e.g., use of observation information in the teacher evalua- 
tion process) as the issues were raised. They were also, 
able, on the spot, to counter barriers to Implementation 
that principals raised. 

e In 1982-83 in the Pennsylvania district, the principals from 
the five field test schools, central office staff, and BSC 
linker shared responsibility for delivering training to the 
remaining twelve principals. This strategy provided the 
original five principals with increased understanding of the 
improvement system. It also gave them an opportunity to 
answer their colleagues' charges that the training could not 
be implemented. The testimony of the five field test 
principals helped overcome the resistance of the new princi- 
pals more effectively than the external linker's arguments. 
Linkers were amazed to see some of the original five prin- 
cipals, who had not been the most enthusiastic participants, 
defend the improvement process to their colleagues. 

As noted in the documentation report of the field test (Blester et 

v t 

al.% 1983), central office staff and principals felt relatively confident 
in carrying out their implementation tasks after receiving training. As 
expected, linkers and central office staff felt that principals first 
grasped and put to use the more mechanical aspects of their training. The 
more difficult aspects required follow-up review and opportunities for 
practice. For example, almost all principals felt confident in their 
ability to observe classes for student engagement rate. However, most were 
less confident and demonstrated much less skill in the more complex behaviors 
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such as conferencing with teachers or observing classes for critical lesson 
elements. 

While the principals' need for follow-up and practice can be partially 
attributed to the limited amount of time available for initial skill 
training, it also reflected i some principals' lack of skill and experience 
with more general skills (e.g., planning, training, and conferencing). 
Principals appeared to vary widely in these skills— a fact that central 
office staff readily acknowledged. The New Jersey district supported weak 
principals by providing individual coaching either from the BSC linker or 
from central office staff. However, the Pennsylvania district was less 
able or willing to provide such coaching, and central office staff rarely 
worked with individual principals. The concept of prescribed or differ- 
entiated lnservlce to address the training needs of individual principals 

/ 

was not operative in any of the three districts. 

/ 

Central office staff and principal reactions to tra:* ting were quite / 
positive with respect to the relevance and quality of the training. 

However, most trainees reported that training was too Intense and that ; 

/ 

there were too many materials to be assimilated so quickly. / 

/ 

Training Teachers 

Although BSC staff conducted most initial training of principals, , 
principals trained teachers, or assisted in the training, although hesi- 
tantly. Principals were not accustomed to training teachers; they custom- 
arily brokered out the LUilding lnservlce time to central office staff or 
to outside consultants. Tj support principals in training teachers, BSC 
provided videotapes that conveyed the technical aspects of training. 
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Observations and interviews with principals indicate that BS£-deve loped 

videotapes conveying the program's rationale and the tore technical aspects 

of training were valuable aids in training teachers. BSC linkers also 

reviewed with the principals Materials prepared specifically to guide thei 

training of teachers. In recent implementations, sore tine has been 

allotted during the principals' training to preparation for the teacher 

training session. This seems to be a very helpful strategy for Improving 

the teacher sessions. Linkers and central office staff devised additional 

strategies for supporting principals in training teachers. 

e In order to provide principals with support in training and, 
in some cases, to reduce video equipment demands, the 
Pennsylvania and New Jersey districts paired principals or 
teamed a principal with a central office person. Principals 
in Pennsylvania who were paired in groups of two or three to 
give joint presentations for the teachers reported that this 
teaming was helpful to them, affectively, but they felt that 
the teachers would have been more satisfied if each building 
had conducted its own inservlce. Despite uany principals' 
initial hesitancy to train teachers, a number of principals 
Indicated that training teachers helped them achieve a new 
sense of instructional leadership. 

e In the Delaware district, the BSC linker, central office 
staff, and principals shared the leadership of most teacher 
workshops. The principals had been reluctant to lead 
training during the period of program development. However, 
they were more comfortable with this role after using the 
program for several years. 

Several teacher training sessions in each district were observed and 

BSC Interviews and questionnaires asked for teachers' reaction to the 

quality of the inservlce. As was true for the principal-led orientation 

sessions, the quality of the observed sessions varied widely. Some were 

excellent, some were fine, and a few were poor. The quality of the sessions 

was obviously dependent upon the principals' knowledge of ADL, their 

commitment to the program, and their ability as trainers. Sessions which 
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linkers judged as poor almost always included major deviations from the 
scripted training* For example » at one building the principals showed all 
the videotape segments consecutively, rather than interspersing then with 
hands-on work sessions as intended* In other instances, principals with a 
reputation for poor delivery did well, given the scripted training* One 
teacher in such a situation noted that the inservice was the best session 
her principal had ever conducted* 

Teachers 1 reactions to training varied froa school to school within 
the districts* Interview data suggest that teachers reacted in the manner 
predicted by Gall et al* (1982) — they do not favor a tight linkage between 
inservice education and assessment of student* achievement* Gall et al. 
suggest that this attitude reflects teachers 1 concerns that student 
achievement test results could be used Jo evaluate their performances or 
hold them accountable. This concern was particularly evident during the 
fall 1983 training in the* Pennsylvania district. Not only was there a 
great deal of attention in the national press to teacher evaluation and 
merit pay, but the district was also discussing the development of new 
principal and teacher performance evaluation procedures and was increasing 
the accountability of supervisors. 

Conclusions 

Between 1981 and 1984, several significant changes were made in the 
intents for training and in some of the strategies for delivery of training 

First* experience early in the planning with field test districts 
indicated that training ought to include greater attention to the roles of 
participants (teachers, principals, and central office staff) in 
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implementation, and, specifically, ought to provide more attention to the 
participatory supervision aspects of Implementation. The development 

9 

activities BSC undertook to effect these changes were described In chapter 3. 

Second, BSC still considers top-down turnkey training to be an impor- 
tant and worthwhile intent, but realises that in relatively small districts, 
especially those with few central office staff, It can.be difficult to find 
tlae for central office staff both to be trained and then to train princi- 
pals. Thus, BSC has become more amenable to training central office staff 
and principals simultaneously. In these cases, several planning sessions 
and brief training sessions may be held with only the central office staff. 
BSC has attempted this strategy with several small non-field test districts 
and found it to be workable. However, the extent to which such districts 

* 

will be successful in transferring such training to new principals or new 
central office staff is unknown* 

Other changes made or proposed for training are* in the main, changes 
In strategy. These include: 

• Spreading initial training in the two major components, time 
and contpnt ■ over a two-year period* Implementation of the 
two components is then likewise spread over a two-year 
period* BSC experience with this strategy is limited* 

• Restructuring the training sequence for the content component. 
The training in the content component proceeds smoothly only 
if the district already has in place a curriculum, a match 
between the curriculum and the testing program, and a 
testing program that facilitates diagnosis of class strengths 
and weaknesses* BSC experience has indicated that these 
elements tend not to be in place* The first step in the 
revised training would be a session with central office 
staff to explain the rationale of the content managamtnt 
process and make specific plans for provision of the needed 
data* curriculum* and content matches. Principals 1 training 
will be revised to be similar in scope to the teachers' 
training — making use of the specific curriculum and data 
formats the principals will be using with their teachers. 
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Thi» strategy may prow to work well in conjunction with 
spreading training and Implementation over two years. Ae 
the district works to implement the tine component, it can 
plan and prepare to implement the content component during 
the second year. BSC has recently begun (September 1984) 
working with a number of districts with this strategy. 

Improving existing training activities and training packages 
to (1) reflect the changes in content training describe* 
above; 1(2) reduce the number of district training pieces; 
(3) increase the amount of- trainee participation in 
training; and (4) provide more attention to the process of 
turnkey training (e.g., provide more opportunities in the 
training for demonstrating and having the' trainees practice 
their roles in the training of principals or teachers)* 
Many Ideas and suggestions for these revisions have been 
accrued and a few have been attempted in recent sessions. 
Revision of materials is planned for the coming fiscal year. 

Delivering training to superintendents, In a separate academy. 
The academy would offer very abbreviated training in technical 
skills but stress the central, office decisions and resources 
needed for Implementation. Both the superintendent and the 
external linker would be better prepared to make a decision 
on the appropriateness of implementing AOL in the district 
after the superintendent participated in the academy. 
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CHAPTER FIVE 
IMPLEMENTING, EVALUATING, AMD INSTITUTIONALIZING 

* 

BSC has always assumed chat after completing training In the management 

of classroom variables district staff would need assistance In transferring 

i 

their new anderstandings Into practice. Therefore, BSC (1) provided technl- 
cal assistance during and following Implementation, and (2) encouraged t 
districts to take steps to evaluate and institutionalize their Implementa- 
tion of ADL. This chapter focuses on BSC staff experiences with technical 
assistance, and the effect of technical assistance on implementation, 
evaluation, and institutionalization of ADL. 

The chapter has three sect ion s ^pn e fQt each district. Each district 
section discusses ; 
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e general characteristics of the district and the history of 
Its cooperation with RBS 

• the ways BSC and district staff supported Implementation of 
key program elements, and the extent of that implementation 

• the activities BSC and district staff undertook to forma- 
tive ly evaluate the Implementation 

a the activities BSC and district staff pursued in attempting 
to institutionalize the program. 

The three district reports follow the same outline but vary in the 

amount of detail Included, especially for the two years following the field 

test year. This variation stems from the differences among the districts 

in the amount of leadership assumed by district personnel and the corres- 

ponding role of the BSC linker in those years. In the Pennsylvania district, 

the BSC linker was actively Involved in training additional! principals and 

leading principal seminars. In New Jersey, district leadership assumed a 
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prominent role, and thus the BSC linker provided somewhat less frequent 
technical assistance and coaching. In the Delaware district, changes in 
district lea Jership resulted in very limited BSC contact. In spite of 
these differences each report presents some interesting insights into the 
process of building local capacity, and together the three reports permit 
some interesting comparisons and constrasts. 

The Pennsylvania District 

The Pennsylvania district contrasts with the New Jersey and Delaware 
districts because it is larger (with a 1981 student population of about 
12,000) and was not involved in the development of Achievement Directed 
Leadership. The district, which serves an industrial city and its 
surrounding boroughs, is the fifth largest in the state, and in 1981 - 
Included 17 elementary schools (K-6), 4 junior high schools (7-9), and 2 
senior high schools. (In the fall of 1982, the four junior highs became 
middle schools housing grades 6, 7, and 8.) The district's minority 
population, about 12 percent of the total student population, is predomi- 
nately Hispanic. Prior to the field test, districtwide reading and math 
achievement scores were at the 55th percentile for math and the 50th 
percentile for reading. There was little press for improving achievement. 

The district s history of "loose coupling" was documented in the 1982 
Middle States Association evaluation: 

Prior to the current superintendent, there had been a 
succession of superintendents who served only briefly. One 
of the obvious consequences of this was a diffusion of the 
elementary program and services as principals tended to 
focus on particular needs of the communities served by their 
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respective schools. Good as this aay be for some purposes, 
the consequent blurring of lines to and from the central 
office deprived the schools' of the central leadership and 
curriculum management services vital to the city-wide 
development and monitoring of quality education, (p. 2) 

One example of the impact of this history was that prior to 1980 each 

junior and senior high* math teacher selected his/her own textbook. Also, 

BSC linkers noted that as late as the 1983-84 school year, a variety of 

report cards was used in the district's elementary schools. 

The district agreed to participate in the 1981-82 field test, primarily 

for staff development purposes, with partial implementation of the approach 

in five elementary schools. Key central office staff involved In the 

implementation were the superintendent, the superintendent's administrative 

Intern, the director of instruction, and two curriculum specialists, one 

for reading and one for math. Pour resource teachers were also included in 

the training and seminar sessions. 

« 

In the summer of 1982, plans were developed to expand the Implementa- 
tion to other schools in the 1982-83 school year. However, the only 
implementation activities that actually occurred were the training of the 
remaining 12 elementary school principals and 1 middle school principal and 
the development of plans for implementation in all 17 elementary schools. 
The district leadership exerted virtually no press for immediate district* 
, wide implementation or for active continuance of the approach in the five 
field test schools. 

In the fall of 1983, the superintendent gave the district's new 
director of instruction responsibility for implementation districtwlde at 
the elementary level, but did not give him the line authority over princi- 
pals necessary to mandate implementation decisions and procedures, k\\ 

/ 
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elementary schools were involved in the implementation in 1983-84. However, 
schools varied in their degree of implementation of program elements. 

Technical Assistance to Build Capacity 

The following discussion of technical assistance is organized around 
key program implementation elements — attention to the critical classroom 
variables and the four elements described in chapter 3, Each section 
describes the strategies employed to promote and sustain Implementation of 
the program element along with the extent to which implementation actually 
occurred * Table B-l in Appendix B gives a summary of the interview and 
questionnaire data relevant to the extent of implementation of four of the 
elements — attention to classroom variables, principal seminars, principal/ 
teacher conferences, and superintendent/principal conferences. 

Attention to classroom variables , ADL gives special attention to four 
classroom variables: prior learning, student engaged time, coverage of 
criterion content, and academic performance ♦ BSC technical assistance to 
the district over the course of the three years was directed primarily 
toward providing, or helping the district provide, principals with the 
information necessary to monitor content coverage, prior learning, and the 
ncademlc performance variables. ^ 

Early in the 1981-82 field test year, BSC staff supplied the district 
with the text to achievement test match for two math basals and with the 
Item domain descriptions for the reading portion of the achievement test* 
These documents gave principals a means of helping teachers plan their 
coverage of math content and a means by which principals and teachers could 
monitor that coverage. Not until verV late in the school year was BSC able 
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.to supply a text/ test natch for two reading basal s. Although BSC develop- 
merit of the text/test Batches was a heavy Investment of technical assis- 
tance, it is unlikely that principals and teachers would have attended to 
content coverage in the 1981-82 implementation without the matches. 

In the field test year, the five principals monitored student engaged 
time in both math and reading. They also monitored content coverage 
primarily in math and to a lesser extent in reading. BSC staff used class 
lists supplied by the principals and district test records to compute 
each math and reading class's general achievement level. Principals were 
asked to share this information with teachers and help them set appropriate 
goals. Little attention was given to specific prior learning strengths and 
weaknesses since appropriate test data were not available. BSC Interviews 
and BSC staff experience indicate that the- academic performance variables 
(success, mastery, and review) were not consistently attended to, and in 
many cases, standards (e.g., what constitutes mastery) varied from teacher 
to teacher and from school to school. 

In 1982-83 BSC and district staff concentrated on training all 17 
elementary school principals. BSC staff instructed all principals oi. how 
to perform a text/ test match and assisted them in matching the district's 
new, widely-used math basal to the achievement test. This content match 
provided the information necessary for teacher planning and monitoring of 
coverage in math. ^^' s tec ^uical assistance successfully enabled the 
district to carry out an activity for which BSC had assumed responsibility 
the previous year. Without this technical assistance it is unlikely that 
the content match would have been completed. 
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During the 1982-83 school year, three of Che five principals previously 
trained reported paying little formal attention to the classroom variables. 
However, they did note that when they conducted classroom observations they 
had an increased awareness of certain factors, such as student engaged 
time. They also reported having informal discussions with teachers about 
the critical variables. The fourth principal helped teachers with school 
year planning based on the text/test matches but did not systematically 
monitor student engaged time. The fifth principal gave systematic attention 
to the coverage and time variables and, to the extent allowed by available 
data, to student prior learning. In addition, a number of the newly 
trained principals put aspects of their training into use during the 
1982-83 school year, although implementation was not required. For example: 

• One principal reported using the engagement rate form in 
routine classroom observations. 

• The district provided principals with copies of the test 
item domain descriptions and text/test matches as they were 
completed. Most principals shared these wi^ their teachers 
and encouraged teachers to use the informa to pace their 
instruction. 

Although the technical assistance provided by BSC and the district 
seemed to focus principals' attention on the critical classroom variables, 
relatively little systematic attention was paid to the cluster of critical 
variables in 1982-83. This is not very surprising, given the district's 
lack of press for implementation. 

During the 1983-84 school year* most of the technical assistance 
provided by BSC and the district was, although related to the classroom 
variables, directed at one of the other four program elements and therefore 
is reported later in this section. During this school year, the district 
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required all 17 principals to conduct three sets of classroom observations 
and three principal/teacher conferences. Principals and teachers attended 
to student engaged time in math and reading and to coverage in mathematics. 
For the first time,* teachers were expected to complete year- long plans for 
their math classes and for one of their reading groups. However, teachers 
were asked to prepare year-long plans in reading without the benefit of 
text/test matches. (This lack of matches was due to (1) the large number 
of reading basals being used; (2) the reading curriculum specialist's lack 
of enthusiasm for the project; and (3) the lack of leadership from the 
former director of instruction.) Teachers complained about the amount of 
work involved in planning without such matches. In response, the district 
dropped it* requirements for year-long planning in reading and promised the 
local teachers association that in the future text/test matches Would be 
completed by the district before teachers were required to complete year- 
long plans. Nevertheless, principals were asked to monitor teachers' 
coverage in reading by using a list of test, item domain descriptions 
provided by BSC. End-of-year interviews suggest that systematic attention 
to coverage in reading occured In some but not all schools. When the 
district and BSC had provided the information about the text/test match 
(i.e., math), attention was given to coverage. When this information was 
noi supplied in a readily usable form (i.e., in reading), less attention 
u to the variable, 
by October 1983, the principals had a fairly good grasp of what was 
required to monitor coverage and student engaged time. However, superinten- 
dent/principal conferences suggested that principals' and teachers' atten- 
tion to mastery could be improved. BSC and district staff together planned 
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and led several principal seminars that included discussion of standards 
for mastery and the mastery test* available in the district f s basal texts. 
The director of instruction and the curriculum specialists strongly 
suggested that principals encourage their teachers to use these tests and 
the associated mastery standards, but their use was not mandated. End-of- 
year interview indicated that the technical assistance provided by BSC and 
the district improved the data collection and conferencing of a number of 
the principals* 

On numerous occasions, BSC staff had discussed with the director of 
instruction, the curriculum specialists, and the superintendent the need 
for item analysis data to determine student prior learning strengths and 
weaknesses* The superintendent asked the director of instruction and the 
director of pupil personnel services (responsible for testing) to see that 
teachers received item analyses. The BSC field staff believed that these 
central office staff gave greater attention to the superintendent's request 
tor action fian they might have at other times because they both considered 
themselves candidates for the then vacant assistant superintendent position. 
They had strong personal motivation for demonstrating compliance. BSC 
technical assistance in this effort involved reviewing with the super- 
intendent, the director of pupil/personnel services, and the director of 
instruction the need for the data and the specific plans for its use by 
teachers and principals. BSC personnel also met with the central office 
staff and the test company representatives to clarify the district's 
request for the item analysis data* 

Thus, by the summer of 1984, the district or BSC had arranged for or 
provided data needed for managing the critical variables, with the excep- 
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tion of the text/test match needed to monitor and manage coverage in 
reading. Also, the district had finally agreed to limit reading text 
adoptions to one of four approved basals. In the late spring of 1984, BSC 
field staff shared with the superintendent! the major findings of the BSC 
end-of-year Interviews of principals and a sample of teachers. The inter- 
views indicated that long-range instructional planning was the least 
implemented aspect of the program and that the department of instruction 
had spent little time on ADL. The superintendent charged the director of 
instruction with completing reading text/test matches over the summer. By 
the end of the summer, one of the four district-approved basals had been 
matched to the test and the district reading curriculum. 

P rincipal seminars . BSC intended that central office staff conduct 
regularly scheduled principal leadership seminars. Seminars were intended 
to be problem-solving and planning sessions that would help maintain 
Instructional improvement as a priority and sustain and improve leadership 
practices, 

During the field test year, BSC staff Jed eleven two-hour seminais for 
the five principals implementing the program. The seminars were also 
regularly attended by a district math specialist, the reading/ language arts 
specialist, and also the four district "resource teachers" (whose job 
assignment for the year was the design of the district's middle school 
program). The BSC/district liaison (the district's administrative intern 
who'se major task was to head the middle sehool design group) was always 
present to begin seminars but frequently lei t early to do "other assign- 
ments." The director of instruction attended two seminars; the superin- 
tendent attended one seminar. 
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BSC's planning and delivery of the seminars were crucial to the 
implementation for the year. Early seminars were used to complete 
principal training. Instead of January seminars, BSC staff met 
Individually with principals to answer questions and review completed 
principal/teacher conference forms. Later seminars included problem 
solving for implementation and planning for implementation in 1982-83. 

Each seminar agenda was discussed with the district/BSC liaison, and 
he faithfully carried out the necessary logistical operations. However, 
there were only two instances in which the district evidenced clear leader- 
ship. 

• The superintendent, at BSC's special request, attended a 
portion of the second seminar in October. At that time, he 
distributed and discussed the implementation timeline 
(deadline for rounds of principal/teacher and superin- 
tendent/principal conferences). 

• After receiving evasive answers from principals concerning 
their progress in conducting observations, the district/BSC 
liaison developed (with BSC assistance) and sent out over 
the superintendent's signature an implementation report form 
to be completed by each principal every month and submitted 
to the liaison. 

In 1982-83, seven seminars were held, one per month except during the 
teachers strike (November and December). During the year, the district/BSC 
Liaison, an administrative intern, usually attended seminars only long 
enough to see the meeting begin'. The math and reading curriculum specialists 
continued to attend seminars regularly; the director oJL instruction did. not. - 
attend any seminars. The superintendent's attendance at the February and 
March seminars, again the result of BSC field staff appeals, increased 
principals' attentiveness; however, the change did not last in his absence. 
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BSC staff planned and led the first three seminars vlth only United, 
logistical inp^t from the district liaison. In February, BSC staff began 
planning for seminars with the two curriculum specialists. Agendas were 
designed jointly and the specialists assumed responsibility for portions of 
the seminars. There were several reasons why this strategy for obtaining 
and transferring seminar leadership was only somewhat successful: 

• The director of instruction, the curriculum specialists' 
supervisor, showed very little interest in the program. 

« 

• Curriculum specialists had no line authority over principals 
and seemingly preferred not to assert any proactive leader- 
ship. 

• The reading specialist had some philosophical reservations 
about the AOL model. 

• In April, the math curriculum specialist was assigned to a 
new position and his replacement was not relieved of her 
other responsibilities until the end of June. 

BSC field staff perceived that the lack of district leadership and the 
October 1982 decision not to implement the program with teachers until 
September 1983 lessened principals 1 interest in toe seminars. Nevertheless, 
the attention that a number of principals gave to content coverage and 
student engaged time seems to have been a direct result of the seminars. 
The seminars provided the principals with the conceptual framework for 
implementation the following year, and prompted some principals to call for 
district support of the improvement effort. ' 

During the 1983-84 school year, there were ten hour-long seminars. 
The new director of instruction was initially very enthusiastic about 
establishing his leadership, and during the year BSC was able to transfer 
to him the responsibility for leading seminars, but not for planning them. 
The director of, instruction attended all but one of the seminars and he or 
the curriculum specialists led portions of four of the last five seminars. 

ERIC 63 * 



Although BSC'S role in the conduct of seminars was reduced, BSC staff 
Planning for them was probably critical. Initially, the director of 
instruction lacked the necessary program knowledge. Toward the end of the 
school year, his enthusiasm waned somewhat and he seemed more anxious to 
please than to lead principals. However, he was always responsive to BSC 
inquiries and suggestions and BSC staff were able to sustain the momentum 
of the seminars with his support. 

Principal interest and attention at seminars were heightened by the 
Immediacy of implementation and by the district leadership's Increased 
involvement. BSC interviews and discussions with the principals and the 
director of instruction indicated that implementation problems and concerns 
were addressed not only in seminars attended by BSC but in several other 
principals' meetings, and in phone calls between the director of instruction 
and principals and between newly trained principals and the five field test 
principals. Many principals were concerned that the district leadership 
set clear and equitable expectations for them. The district was in the 
process of selecting one or more elementary schools to close in response to 
declining enrollment, and this seemed to inspire some competitive 
Implementation efforts. 

interview data suggest that principal seminars did help improve imple- 
mentation. For example, the superintendent/principal conferences indicated 
that there wero a number of classes in which the percent of students 
mastering content was low, and that the source of the principals' data was 
often teacher judgment. Consequently, two seminars were devoted to review- 
ing the irportance of mastery and the means of assessing it. Interview 
data suggest that these seminars had an impact on principals' data collec- 
tion and on their conferencing with teachers. 
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Throughout the three years, the seminars were the primary opportunity 
for contact between BSC personnel and principals. As such, seminars served 
as an Informal but regular source of information on the status of 
implementation activities and helped BSC staff in formulating strategies to 
promote implementation. 

Principal/ teacher conferences . BSC intended principal/teacher confer- 
ences to be formally conducted conferences in which a principal and a 
teacher review data on all of the critical student behaviors* identify any 
opportunities for improvement, and agree to improvement plans. The compon- 
ent suggests that principals conference with all teachers at least three 
times a year and perhaps more frequently with teachers with many improvement 
needs. During the summer of 1981, BSC developed a form to record and 
structure these conferences. 

Early in October 1981, the BSC linker met with the superintendent, the 
director of instruction, and the district/BSC liaison to suggest an imple- 
mentation timeline that called for three principal/teacher conferences. 
The timeline was approved by the district leaders and. then shared and 
discussed with principals at an October seminar. During October and 
November, BSC also reviewed the conference form with the district/BSC 
liaison and planned adaptations. For example, since implementation in 
1981-82 did not include teacher-developed year-long plans, questions that 
referred to such plans had to be revised or omitted. The two December 
seminars were devoted to introducing the form and guiding principals 
through BSC-designed case study conferences. 

Early in the 1981-82 school year, the district assigned a resource 
teacher or a curriculum specialist to assist each of the principals with 
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observations or conferences* as needed. However, only two of the resource 

teachers assisted principals with classroom observations (one or two 

observations each), and the reading curriculum specialist assisted only one 

principal who fas having difficulty with conferencing. This low level of 

district assistance can probably be accounted for by the following facts: 

e the schools were relatively small (6-8 teachers) 

e the curriculum specialists' supervisor (the director of 
instruction) showed very little interest in the improvement 
effort. The specialists responded to principals' requests, 
but they were not proactive in addressing principals' needs 

e the resource teachers' supervisor (the BSC/district liaison) 
wanted the resource teachers to attend to their primary 
responsibility— design of the middle school program. 

Questionnaire and interview data indicate that almost all teachers had 
three conferences with their principal during the 1981-82 school year. 
Principals and teachers reported using conference forms, identifying needs, 
and discussing improvement strategies. Three of the four principals who 
held princlpalships before September 1981 reported that prior to 1981 they 
had not conducted such structured conferences with teachers. Of the twelve 
teachers who were interviewed by BSC staff, nine indicated that the confer- 
ences were helpful to them in identifying needed changes and selecting 
change strategies. 

In the 1982-83 school year, only two of the five trained principals 
reported holding formal conferences with teachers. One principal used the 
conference form in two conferences with each teacher and In additional 
conferences with selected teachers. The second principal conferenced only 
once with each teacher and attended to content coverage, prior learning, 
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and . academic performance; but he did not collect data on or discuss student 
engaged tine. 

In the 1982-83 school year, BSC assistance directly related to princi- 
pal/teacher conferences was provided during the spring seminars. BSC staff 
pointed out the Principals Handbook materials on conferencing and guided 
principals through two BSC-designed conference case studies. The district 
leadership did not provide the five field test principals with an implemen- 
tation timeline, or with any specific expectations for implementation. 

In 1983-84, the first year in which all 17 principals were expected to 
implement th* program with teachers, BSC again proposed to the superinten- 
dent an implementation timeline. At the same time, BSC shared with the 
superintendent interview data indicating that without district expectations 
only two of the five principals made a concerted effort to continue imple- 
mentation in 1982-83. This information helped to convince the superinten- 
dent of the need for sharing clear expectations. The new director of 
instruction agreed to the timeline (with minor changes) and decided to use 
a variation of the district report form (developed in 1981) to monitor 
principals 1 classroom observations and conferences. The district agreed to 
three of the four sets of classroom observations/conferences proposed by 
BSC and encouraged principals to conduct additional conferences with 
selected teachers. As noted above, principals were very concerned in 
1983-84 with issues of equity and compliance. While many principals 
supervised only 6-8 teacher^* three or four principals supervised as many 
as 18 teachers. District guideline? asked principals to include every 
teacher in at least one set of observations/conferences, to include the 
vast majority of teachers in, three rounds, and to spend at least 25 percent 
of each day observing or conferencing with teachers. 
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The curriculum specialists and director of instruction supported 
principax/teacher conferences by answering principals' questions during 
informal Meetings and telephone conversations. Central office staff indi- 
cated that most of these conversations included requests for improvement 
strat^cs. Although it was clear to central office staff that principals 
varied in their ability to conference with teachers, BSC knows of no pro- 
active support on the part of the central office staff to help principals 
improve their conferencing skills. 

BSC and district support for conferences can be assessed by conference 
frequency and the reported success of the conferences. 

• All 17 principals conducted conferences with teachers. 
Eleven of the seventeen principals conferenced with all 
regular classroom teachers. Nine of the eleven had three or ^ 
more conferences with all of their regular classroom 
teachers. 

• Twenty of the thirty-four teachers Interviewed said that 
conferences helped them with classroom management and gave 
the interviewer specific examples of changes they made in 
classroom management. Fourteen of the thirty-four teachers 
said that the conferences helped them improve the quality of 
their instruction and were abxe to give the interviewer 
specific examples. 

Interview data from the curriculum specialists, the principals, and a 
sample of teachers suggest that for most principals three structured confer- 
ences around specified, measurable variables, represented a vast change f 
from previous years. 

Overall, it is clear to BSC that the implementation was sustained only 
when there were specific district expectations for principal/teacher 
conferences and systematic monitoring of them. Interview data suggest that 
the structured conference form gave principals a focus for their observa- 
tions, conference discussions, and plans for change. 
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Superinte ndent/principal conferences . The ADL leadership plan calls 
for the superintendent to hold formal supervisory conferences with princi- 
pals to review classroom data and to discuss improvement opportunities. 
The plan calls for at least two such conferences during a 'school year. 

The implementation timeline suggested to the superintendent in October 
1981 called for two rounds of superintendent /principal conferences, one in 
March and one in May. BSC staff described the conferences and reviewed the 
existing form with the superintendent. He agreed to conduct the conferences 
with the five principals. 

The timeline for these conferences was distributed to the principals 
during an October seminar at which the superintendent was present. During 
an early February seminar, BSC field staff reviewed the purpose of the 
superintendent/principal conferences and explained how to complete a 
summary of school data for review during the conference. Principals seemed 
surprised by the idea of a conference with the superintendent and asked a 
few questions. It appeared that several principals had not planned on the 
conferences actually occurring, and had not collected all the needed data. 
BSC also prepared guidelines for the superintendent and tutored him in the 
use of the superintendent /principal conference form. 

In March, the superintendent conducted conferences with each of the 
principals. During end-of-year Interviews, most principals reported that 
although the superintendent demonstrated some discomfort with the form, the 
conferences stimulated their first substantive discussion with the superin- 
tendent about instructional issues. 

The conferences fulfilled the intended purposes: information flowed 
from the classroom to the district level, the superintendent's interest In 
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implementation was demonstrated , and principals were prompted to observe 
classrooms and to conference with teachers* However, conferences did not 
result in the Intended district support to principals or teachers* Also* 
the proposed May conferences were never held* 

In the 1982*83 school yea**, no superintendent/principal conferences 
were scheduled as school level implementation was not required* 

In September 1983, BSC met with the superintendent and the district 9 s 
new director of instruction to discuss Implementation concerns* This year, 
the superintendent raised the need for an Implementation timeline* The 
superintendent and director of instruction made minor changes in a timeline 
the linker suggested* The superintendent accepted the BSC suggestion that 
he portray implementation as a district effort by presenting the timeline 

at a principals 9 meeting that BSC would not attend* Unfortunately, the 

» 

memo distributed at the meeting included references to RBS along with the 
timeline — thereby not placing ownership squarely on the district* 

The 1983-84 timeline Included two rounds of superintendent/principal 
conferences, one in late January and one in late April* Prior to the first 
round of conferences, BSC prepared and led a segment of a principal seminar 
to help principals prepare for the conference with the superintendent ♦ 
Specific instructions were given foj: completing the data collection section 
of the conference form. The BSC linker also reviewed the conference form 
(revised since 1981) with the superintendent and ""rector of instruction* 
The linker encouraged the superintendent to review with each principal the 
past several years' student achievement data and to ask each principal for 
his/her school achievement goals for the coming year* Curriculum specialist 
in the district have routinely prepared five-year longitudinal reports on 
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student achieves organized by grades within buildings. The primary use 
of the reports has been for school board presentations, but the linker 
urged the superintendent to use the* at the conferences in order to 
focus principals' attention on the intended outcome of the Improvement 
effort. The superintendent; 

• charged the director of instruction with the scheduling of 
conferences for all 17 principals during the remaining two 
weeks of January. 6 

• ^ ar ?! d ^ the , dlre ! t0r ° f with (1) reviewing the 
longitudinal math and reading data for each school; (2) 
attending the conferences; and (3) assuring that principals 
came to the conferences prepared, with the required data and 
with some of their own ideas about their school needs and 
goals. 

The director of instruction carried out those tasks, and later he and 
the superintendent conferenced with each of the principals. 

• All 17 principals reported that these conferences repre- 
sented a major change from past practice. (The five field 
I*?* f^JSif* 1 ; acknwled « ed that a similar conference was 
1982 or in 1983 ) that "° COnference was held P rlor to 

* Durin « end-of-year interviews, about one-third of the. prin- 
cipals felt that the conference was a mechanical account- 
ability check, whereas the other two-thirds felt that the 
conference was worthwhile. BSC interview results suggest 
that those principals that seized the opportunity of having 
the superintendent's ear, and spent time carefully preparing 
their case" felt that the conference was valuable. 

• Four principals indicated that instructional issues they 
perceived as not directly related to ADL (e.g*, class 
size/teacher distributions) were also discussed at the 
conferences. In two instances the principals reported 
success in obtaining requested instructional resources. 

The second set of superintendent/principal conferences scheduled for 
April 1984 was never held. However, the one round of conferences that was 
conducted seemed to; 
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• proapt classroom observations and conferences 

• convey data from the classroom to the district level 

• demonstrate the superintendent's interest and commitment 

• identify one significant opportunity for improvement common 
to a number of buildings: the lack of uniform sources and 
standards for mastery and the relatively small amount of 
attention given to mastery. 

Differentiated inservice . If district reform and renewal are to take 
place, the opportunities for improvement identified in seminars, principal/ 
teacher conferences, superintendent /principal conferences, and informal 
discussions must be addressed. The leadership plan suggests that the 
district office and principals be proactive in providing for needed small 
or large group inservice based on diagnosed needs. 

In the Pennsylvania district, there has been little differentiated 
inservice from the district level. During the field test year, all 
differentiated inservice provided to principals was in the form of 
technical assistance from the BSC linker. For example, a BSC linker 
assisted two principals in presenting to their faculties the rationale for 
work with the content variables. On another occasion, the linker suggested 
that a principal was in need of special assistance with the conference form 
and that one of the curriculum supervisors provide the needed coaching. 
The director of instruction replied that the principal would have to make 
do with the same level of assistance as each of the other four principals . 
On one ocassion district staff agreed that principals could benefit from a 
review of the district's standardized testing procedures and an analysis of 
achievement data. The district test coordinator held this review for all 
staff involved in the ADL, implementation. The linker helped the testing 
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coordinator Integrate his presentation with the work done in ADL, using the 
opportunity to* infona the coordinator of the goals and objectives of the 
improvement effort. 

In 1982-83, when the district emphasized training of the 12 remaining 
elementary principals, neither BSC nor the district provided differentiated 
inservice. The principals elected to have their routine district inservice 
be a presentation on a clinical supervision model compatible with ADL. BSC 
staff attended the principals' inservice, and pointed out where the two 
models were compatible. 

In 1983-84, the new director of instruction had a more positive 
attitude toward differentiated inservice. He reported helping principals 
with aspects of their implementation in numerous phone calls and short 
Informal meetings. In response to parental concerns about one teacher, the 
district had the two curriculum specialists (as well as the school princi- 
pal) observe the teacher and provide individual assistance. They reported 
using strategies contained in ADL materials to help the teacher with 
classroom management. 

The 1984 end~of-year interviews with principals indicated that during 
the superintendent/principal conferences the superintendent had informed 
each principal that the curriculum specialists would respond to any requests 
for assistance or support* Many principals commented that they never 
requested such support as it was not needed. Although centra] office staff 
were aware of some principals 1 opportunities for improvement, the primary 
mode of assistance was one of responding to requests. Similarly, the 
director of instruction balked at officially "excusing" the more skilled 
principals from any of the seminars. Central office staff seemed to fear 
that differentiated treatment would lead to dlssention. 
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While the district was responsive to widespread opportunities for 
total group lnservlce (e.g., the seminars devoted to mastery and to the 
testing system), the district has not provided for tailored lnservlce for 
smaller groups of principals or teachers who share common opportunities . 
Since principals vary widely in their prior knowledge and skill, the design 
of seminars for the total group has been difficult, and the district has 
been hesitant to provide a variety of activities to match the variety of 
improvement opportunities* Although able principals can be the source of 
ideas and strategies for the weaker principals, this role becomes difficult 
over time. The BSC linkers continue to advocate differentiated lnservlce 
in spite of central office reluctance* 

Technical Assistance for Evaluation 

Plncus and Williams (1979) suggest that innovators must keep their eye 
on the primary goal or intended benefit of an Innovation* While secondary 
benefits (e.g., Increased morale, recognition) may be reinforcing, there Is 
a danger that these secondary benefits may distract educators from the 
primary goal. From the outset, BSC Intended to provide technical assistance 
for evaluation to assure that districts focused on the primary benefits — 
improved instruction and increased student achievement. While BSC resources 
prevented the systematic collection and analysis of data on changes in 
Instruction (classroom conditions) before and after implementation, data 
were gathered on participants 1 perceptions of changes in their roles — 
especially the extent to which such changes reflected the leadership plan* 
Data were also gathered, by both BSC and the districts, on student achieve- 
ment. 
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Role changes . One goal of ADL is for district staff to carry out 
activities defined in the leadership plan. If staff alter their behaviors 
to conform nore closely to those specified in the plan, the program has had 
an impact. The discussion of role changes of educators in this district is 
based primarily on interviews BSC conducted with central office staff, 
principals, and teachers. (A complete interview schedule is in Appendix 
A.) An interesting concomitant benefit of the BSC interviews was that they 
prompted central office staff, principals, and teachers to reflect on their 
leadership activities. In some cases, this reflection led the interviewee 
to establish his or her own goals for improvement. The district carried 
out some information gathering activities, although these were less struc- 
tured and involved a smaller sample of individuals. In a few cases, BSC 
staff assisted the central office staff with their data collection. The 
following discussion of perceived role changes is organized by levels — 
superintendent, central office staff, principals, and teachers. It con- 
cludes with a review of how BSC helped the district use the collected data 
tor planning. 

In the spring of 1982, the superintendent reported that the program, 
specifically the superintendent/principal conferences, had Increased his 
communication with principals on issues related to instruction. He noted 
that he would like to expand the conferencing procedure to all principals. 
The superintendent aUo reported that involvement with ADL had increased 
the amount of inservlce the district provided on instructional leadership. 
The two curriculum supervisors felt that the superintendent's role had 
changed as a result of the program. One stated that the superintendent had 
become a much stronger Instructional leader, whereas the other supervisor 
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felt that the superintendent demanded greater accountability from prin- 
cipals. Most of ^ne principals indicated that the superintendent's role 
changed a great deal or somewhat. The change most noted was the superin- 
tendent's conferencing with principals about instruction. 

The 1983-84 director of instruction, though new to the district, felt 
his role had changed somewhat in that he would have acted differently if he 
had not been involved in the program. Specifically, he would have given 
less attention to instructional pacing and coverage, daily success, and 
year-long instructional planning.-, One curriculum supervisor stated that 
her role had changed somewhat, and noted planning for content coverage as a 
major change. The other curriculum supervisor stated that she had changed 
very little. She simply focused in a different way on her usual activities. 
Although teachers varied widely in their opinions of central office staff 
role changes, many felt unable to make judgments as they had little commun- 
ication with central office staff. 

Most principals felt their own role as instructional leader had 
changed either somewhat or a great deal as a result of their involvement 
with the program. Several principals reported more direct contact with 
teachers and students, more awareness of the critical classroom variables, 
and more regular observation of teachers along more specific dimensions. 
The 1983-84 director of instruction perceived that principals had changed a 
great deal, especially in their new emphasis on specific classroom variables 
and increased regard for long-range instructional planning. During a 1983 
planning meeting, the director of testing reported that the five principals 
involved in the field test had increased their use of test data for instruc- 
tional planning. One of the curriculum specialists also noted that 
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principals had bacon* acne Involved in instruction, had Improved their 
classroom observations, and vera now using test results to identify strengths 
and weaknesses. However, the other curriculum specialist was unsure 
whether the principals 1 roles had changed. In both the 1982 and 1984 
Interviews, most of the teachers reported that the principals' roles had 
changed somewhat. The change most commonly noted was principals' classroom 
observations of student engaged time. 

The district also assessed principals' performance of some Instruc- 
tional leadership activities. 

a In the summer of 1983, the director of instruction and 
acting assistant superintendent Interviewed each of the 
principals involved in the 1981-82 field test about their 
use of ADL in 1981-82 and 1982-83. Information gathered was 
used to make decisions about the 1983-84 implementation. 

a In October 1983, the director of instruction visited a 

number of the schools to assess the principals' training of 
teachers. The information was used to help principals plan 
for a second scheduled day of teacher training and to make 
additional decisions about the 1983-84 Implementation. 

i 

f Later in October, central office staff and BSC linkers 
visited principals, and in some cases a few teachers, to 
obtain reactions to training in both managing instructional 
content and managing instructional design* District and BSC 
staff used the information to plan some of the principal 
seminars and formulate decisions about district 
implementation requirements. 

Most teachers agreed that their own role as instructional leader 

changed somewhat as a result of the program. The changes most frequently 

cited were: increased awareness and management of student engaged time, 

Increased attention to content coverage, and completion of year-long 

instructional plans. The director of instruction, curriculum supervisors, 

and principals all agreed with teachers 1 assessments. They also emphasized 

teachers' increased awareness of the critical classroom variables, and 

teachers 1 improved year-long instructional planning and classroom management. 
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Each spring , the BSC linker carefully prepared a summary of the inter- 
view results and developed some tentative implications for the district's 
future planning. The report was shared with the superintendent and director 
of Instruction, and ensuing discussions focused on how the district could 
use the Information to Improve its instructional leadership. In some cases 
the superintendent volunteered some action prior to any BSC suggestion* 
For example, the 1983 interview data and the district's own discussions 
with the field test year principals, prompted the superintendent to assign 
the director of Instruction responsibility for the ADL implementation. In 
other instances, BSC suggested some district action. At times these 
suggestions were readily accepted and acted on (e.g., establishing a 
timeline for key implementation events); at other times the suggestions 
were accepted but action was delayed (e.g., preparation of /school year 
planning guides for all adopted reading texts). A principal seminar at the 
end of each school year Included an Informal presentation of interview data 
that were then used by the principals in developing goals and implementation 
suggestions for the following year* 

During the three years, the linkers shared with district personnel how 
information gained from conferences, discussions, or observations could be 
used to influence planning or seminar agendas. For example, during a 
number of the 1983-84 seminars, it became obvious to the BSC linkerr that 
principals were not using the "Instructional Events Checklist" during 
classroom observations. BSC linkers and district office staff then planned 
a number of seminars to re-explain and provide practice with the form. 

Student achievement . The primary goal of ADL is improvement of 
student achievement «. Throughout the implementation, the BSC linkers 




attempted to keep the participant* focused on the extant to which this 
benefit was being attained. Despite the lack of public or overt district 
press for improved achievement, the superintendent always considered 
student achievement to be a crucial outcome and consistently directed 
central office staff to prepare presentations on student achievement for 
the school board. 

Each year. BSC collected grade level achievement test data for each 
elementary school. Although BSC staff analyzed data. to relate achievement 
at the building level to the degree of implementation at the building 
level, only district level data are presented in this report. Scores from 
1980-84 are presented in Table 1 as Normal Curve Equivalents (NCEs) , along 
with gains from 1980-81, 1981-82, 1982-83, and 1983-84. (See Appendix C 
for a discussion of the significance of achievement gains.) 

In the two years before the district first undertook implementation of 
ADL, students at most grade levels vere scoring around the national average 
(50 NCEs) in reading and math. Some grades registered moderate gains from 
\ 1980 to 1981. in reading, the gains were, for the most part, balanced by 
decreases. The average change in reading was +0.7 NCEs; the average change 
\ in math was +1.8 NCEs. Changes ranged from -1.6 to +2.6 NCEs in reading, 
and from 0.0 to +4.3 NCEs in math. 
\ The district decided to implement certain elements of ADL in five 

elementary schools in the 1981-82 school year. After one year of implemen- 
tation, test results showed the continuation of the positive gains in math 
achievement, but again, the losses in reading seemed to balance out the 
gains. The average gain in reading was +0.3 NCEs, and in math, +2.3 NCEs. 
Changes ranged from -1.6 to +2.6 NCEs in reading and from 0.0 to +4.0 NCEs 
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Table 1 

Student Achievement Results: 1980-1984 
Pennsylvania School District 



READING 





Scores 3 


— — , — , f 

Gains/Losses 




Grade 


















83-84 




1980 


1981 


1982 b 


1983 C 


1984 


80-81 


81-82 j 


82-83 


1 


48.7 


51.1 


51.6 


55.9 


58.7 


+2.4 


+0.5 


+4.3 


+2.8 


2 


47.9 


50.5 


51.6 


55.3 


58.3 


+2.6 


+1.1 


+3.7 


+3.0 


3 


51.6 


50.0 


52.6 


54.9 


59.3 


-1.6 


+2.6 


+2.3 


+4.4 


4 


51.1 


50.5 


49.5 


54.2 


58.0 


-0.6 


-1.0 


+4.7 


+3.8 


5 


54.2 


54.8 


53.2 


59.9 


61.1 


♦0.6 


-1.6 


+6.7 


+1.2 


Mean 


50.7 


51.4 


51.7 


56.0 


59.1 


+0.7 


+0.3 


+4.3 


+3.1 



MATHEMATICS 





a 

Scores 


Gains/Losses 


Grade 


















83-84 


1980 


1981 


1982 b 


1983 C 


1984 


80-81 


81-82 


82-83 


1 


52.6 


55.3 


59.3 


64.9 


62.3 


+2.7 


+4.0 


+5.6 


-2.6 


2 


52.1 


56.4 


58.7 


62.3 


63.8 


+4.3 


+2.3 


+3.6 


+1.5 


3 


52.6 


53.2 


57.0 


58.1 


tf2.6 


+0.6 


+3.8 


+1.1 


+4.5 


4 


50.5 


50.5 


52.1 


55.9 / 


59.8 


0 


+1.6 


+3.8 


+3.9 


5 


55.9 


57.5 


57.5 


59.9/ 


68.7 


+1.6 


0 


+2.4 


+8.8 


Mean 


52.7 


54.6 


56.9 


60^2 


64.4 


+1.8 


+2.3 


+3.3 


+3.2 



a 



Scores represent Science Research Associate (SRA) test results as NCEs. 



ADL 



was first implemented^ in the district in the 1981-82 school year, but in 



only five of che seventeen elementary schools, 
°ADL implemented in all 17 elementary schools. 
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in nath. It should be remembered that only 5 of the 17 elementary schools 
implemented elements of the program. This level of effort seemed to exert 
little Impact on achievement for the district as a whole. However, an 
analysis of achievement results of the 5 schools in 1981-82 (Blester et 
al., 1983) shows that the average gain In reading for the 5 schools (+1 
NCEs) was higher than the average for all 17 elementary schools. Similarly, 
the average gain In math for the 5 schools (+6 NCEs) was greater than that 
of the 17. For the five schools, school l-y*i gains In reading ranged from 
-2 co +5 NCEs, and In math from 0 to +12 NCEs. The greater gains in math 
are consistent with the greater level of program implementation (especially 
content management) in math. And, BSC's analysis also showed that the 
magnitude of the increases varied with the degree of implementation at the 
school level. 

Over the course of 1982-83, BSC staff trained all 17 elementary school 
principals in ADT, and planned with them for the next year's Implementation. 
Interviews conducted in 1983 indicated that many of these principals put 
aspects of the program (especially attention to coverage) into practice in 
1982-83. In the spring of 1983, all grades scored substantially above the 
national average in reading and math. The gains in reading achievement 
increased substantially, with an average change of +4.3 NCEs. The general 
upward trend in math achievement gains intensified at most grade levels, 
with an average change of +3.3 NCEs. No grades registered losses. Changes 
in reading achievement ranged from +2.3 to +6.7 NCEs; changes in math 
achievement ranged from +1.1 to +5.6 NCEs. The actual cause of these 
improvements in achievement is unknown. However, it can be hypothesized 
that the principals* training in ADL had a positive impact on instructional 
leadership (see previous section) and this resulted in achievement gains. 



In 1983-84, ADL was implemented in all 17 elementary schools. Teachers 



several rounds of observations and conferences. Student achievement from 
the spring of 1984 showed a continuation of the positive trends. Achieve- 
ment in math increased at about the same rate as in the previous year. The 
average change was +3.2 NCEs, with grade level changes ranging from -2.6 to 



i *ase than the previous year. The average change was +3,1 NCEs, with 
^ i.l 'evel changes ranging from +1.2 to +4.4 NCEs. 

In brief, student achievement in the Pennsylvania district was around 
the national average and increasing slightly, before implementation of ADL. 
Partial implementation of the program in approximately one-third of the 
district's elementary schools seemed to have a positive influence on 
achievement la those schools. Achievement gains in the five schools were 
greater In math than in reading, which was consistent with the greater 
emphesis on content management in math. Student in the district began to 
Improve noticeably in the 1982-83 school year. 

Each summer BSC linkers shared the results of their analysis of 
student achievement data and Its relationship to implementation with tht> 
superintendent. Although It was clear that the superintendent had solicited 
principal and central ofcfice opinion about the program, his decisions 
regarding implementation seemed to be strongly influenced by student 
achievement outcomes. The superintendent always kept the set ool board 
informed about Implementation *nd achievement. He requested copies of 
BSC's report on the Pennsylvania district field test and on subsequent 
years' analyses of achievement test results for the board. 




principals worked with them through 



+8.8 NCEs. Achievement ifa, reading also increased but at a lower rate of 
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The last principal seminar of each school year typically included time 
for principals to compare their last year's student achievement scores* the 
present year's scores* and the goals 'they had set for the present year. 
Principals were presented data in a f'orra which allowed them to compare 
grade level data (i,e., this year's first grade vs. last year's first 
grade) and also the growth of student cohorts (i.e., this year's second 
grade vs. last year's first graded . The relationship between implementation 
and achievement was also discussed* Most principals seemed to value these 
discussions. A few raised the question — how long can we be expected to 
continue improving? 

Technical Assistance and Institutionalization 

Most theories which describe the stages of an innovative process 
(Herman & McLaughlin, 1975; Hage & Aiken, 1970; Rosenblum 4 Louis, 1981; 
Yin & Quick, 1978; Zaltman, Duncan, & Holbek, 1973) agree on a final stage 
which follows Implementation. This stage is characterized by continuation, 
routinization, or institutionalization of the innovation. Both BSC and the 
districts adopted specific strategies to build ADL practices into district 
routines. 

The early signs of institutionalization in the Pennsylvania district 
are mixed. Many of the positive signs are the result of the initiative of 
several individuals, and thus may disappear if the individual advocates 
were to leave the district. 

In addition to tae design features of ADL that were intended to 
enhance institutionalization (e*g., relatively low cost, development of 
local capacity, no provision for funding to the district), linkers pursued 
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other strategies to sustain Implementation and foster institutionalization. 



One strategy was to keep the superintendent up-to-date on the status of 
implementation: 

9 In 1981-82 and 1982-83, the linker sent monthly letters to 
the superintendent to inform him of implementation concerns 
and nf ogress.' This strategy was not employed in the 1983-84 
school year as the linker felt that the new director of 
instruction would assume this responsibility. However, the 
director of instruction was distracted by other assignments 
and his communication with the superintendent did not 
include any long-range planning. Nothing seems to substi- 
tute for direct communication with the superintendent. 

A second strategy was to increase district staff's commitment to and 

understanding of instructional leadership: 

• The superintendent and director of instruction were fre- 
quently counseled and provided literature on practices that 
lead to effective instruction " (e.g. , use of curriculum 
mastery tests). This strategy was sometimes successful, 
but the changes in behavior were gradual. 

* Another strategy was to encourage processes important to institutional iza 

tion (e.g., the incorporation of new procedures into policy and routine): 

• The linker continually sought to tie the leadership plan to 
other district initiatives and interests. For example, each 
year the school toard adopted a set of goals. The linker 
always pointed out to the superintendent which goals could 
be supported by the leadership plan. This strategy seemed 
helpful; It usually evoked the superintendent's vocal 
support of AD T - and the subsequent participation of central 
office staff 

• Whenever possible, the linker informed other central office 
personnel of the goals of ADL and the leadership plan. This 
seemed particularly important in the Pennsylvania district 
as communication between divisions (e.g., testing and 
curriculum) was minimal. 

• The linker gradually turned over responsibility for the 
logistical and substantive planning to central office staff 

• The linker encouraged the district to clarify, in writing, 
expectations for principals (e.g., provide ,\ timeline for 
critical events). 
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• The linker counseled Che district to avoid any duplication 
of effort. For example, the linker persuaded the district 
to substitute a set of achievement data useful to teachers 
in planning for a Stet the district received that was not 
useful to teachers, 

m The linker demonstrated to the superintendent how formative 
evaluation could be carried out using achievement test 
results and interview data from central office staff, 
principal, and teachers. The linker also suggested how this 
information could be used in future planning. 

The district also pursued some of its own strategies for facilitating 

or spreading various elements of the improvement effort. For example: 

• In 1983, the district reduced the number of approved reading 
basals from fifteen to four, thus making alignment and 
monitoring more feasible, 

• The district extended training to staff only peripherally 
Involved in the implementation. For example, in 1981, the 
superintendent included the four resource teachers assigned 
to plan the district f s new middle school program in the 
principals' training sessions and seminars. He felt that 
the^jraining would be helpful to them in their design work. 

Other strategies had the effect of placing responsibility or authority for 

Implementation within the district's permanent organizational structure. 

For example: 

• In the fall of 1983, the superintendent placed responsi- 
bility for the ADL effort with the director of instruction. 

• In 1981-82 and 1983-84, the district developed and required 
principals to submit monthly reports on the number of 
observations and conferences conducted. 

Some strategies both spread the approach and placed it into the 

organization's authority structure. One example is the superintendent's 

successful push to have the resource teacher position in the middle schools 

reclassified from a non-supervisory position to a supervisory position. 

This provided the additional supervisory personnel needed for the relatively 

large middle school (approximately 65 teachers), and also assured that the 

leadership carried the necessary authority. 
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Among the more discouraging signs concerning Institutionalization Is 

a 

the low level of implementation of superintendent/principal conferences and 
differentiated inservice. Another sign which does not bode well for 
institutionalization is the wavering of the district leadership's attention' 
to implementation. Finally, the lack of proactive action and commitment on 
the part of a few district leaders is also discouraging. 

There are promising signs of institutionalization* One is the spread 
of ADL elements to subject areas other than reading and math and to grade 
levels other than the elementary (K-5) level: 

# One elementary principal reported conducting principal/ 
teacher conferences that address the four critical student 
variables in the areas of science and social studies* as 
well as In math and reading/ language arts. 

• One middle school principal (formerly the 1981-82 district/ 
BSC liaison) reported successfully helping some of his 
weaker teachers plan for the year by having them attend to 
content coverage , prior learning, and the days needed to 
cover topics. » 



• The current district math and science curriculum specialist 
required science textbook companies to submit content 
matches between their texts and th6 district's achievement 
test. She also ve^^ied the accuracy of these matches prior 
to having a committee select a series for dlstrictwide use. 



• The same specialist helped teachers piloting the selected 
science text plan by allocating days in the school year to 
text topics. She is now developing additional information 
to be included in guides for all teachers f year-long 
planning. 

# This curriculum specialist also required senior high math 
and science chairpersons to document the alignment of their 
departmental final exams with the course objectives. 

Additional promising signs are the inclusion of elements of the leadership 

plan in district policy documents and the incorporation of aspects of the 

plan in routine district operations: 
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• In response to a suggest ioa from the Middle States Associa- 
tion, the district is preparing a monitoring system for the 
reading program. The sys%?» specifies the responsibilities 
of central office staff* principals, reading specialists, 
and teachers and has been integrated with the planning and 
monitoring requirements of the ADL leadership plan. BSC 
staff were invited to review the plan apd ensure its compati- 
bility wlrii ongoing work in ADL. 

• The district presented principal? with written expectations 
for conferences and observations in the 1983-84 school year. 



e During a 1983-84 lnservice workshop, math teachers at one of 
the four middle schools were required to develop a "curriculum 
map." (Curriculum maps are a local name for year-long 
planning guides that attend to coverage and prior learning.) 

• During the 1983-84 school year, the math/science curriculum 
specialist assisted all 7th grade math teachers in differ- 
entiating between essential and enrichment textbook objec- 
tives* Teachers were also given a recotmaended pacing guide 
for instruction in the essential skills. This guide was 
developed to assure attention to coverage of tested skills. 



The New Jersey District 

The New Jersey district is a small urban district. In 1981, the 
district served 4,600 students (of whom 89 percent were Black or Hispanic) 
and included eight elementary schools (K-6 and in some cases K~7), one 
middle school (grades 7 and 8), and one high school (grrdes 9-12). The 
district cooperated with RBS in developing ADL for two and a half years 
prior to the 1981-82 field test. This work involved the principals and a 
few teachers in three of the district's eight elementary schools, and 
focused on the development of training materials for the management of 
classroom variables. An elementary supervisor and two elementary resource 
teachers provided support from the ceritral office. 

In the spring of 1981, after havii^j seven superintendents in the 
previous ten years, the district appointed a new superintendent. Low 
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student achievement on the standardized achievement test and state minimal 
competency test had resulted in media and school board press for improvement . 
The new superintendent reviewed all district projects , and found the three 
principals who had implemented portions of ADL to be very enthusiastic 
about the program 1 s impact. In an effort to reverse the pattern of low 
achievement, the new superintendent agreed to district participation in 
the 1981-82 field test of ADL as the focus of a districtwide improvement 
plan. Implementation Included all eight e tementary schools and the single 
middle school. Key central office staff involved in the implementation 
included the superintendent, the assistant superintendent, and curriculum 
supervisors from the Department of Instruction (DOI). 

The following year (1982-83), the structure of most schools changed 
(i.e., one elementary school closed, the middle school became a K-8 elemen- 
tary school, ahd the remaining elementary schools added either a seventh or 
eighth grade); however, administration support of ADL continued. Because 
of the active support of district leadership it was possible for the linker 
to shift attention to the high school. Overall, the district actively 
maintained the program and remained in close contact with BSC. 

In 1983-84, district leaders assumed that ADL practices were firmly 
embedded in district routines and therefore would continue. They eased the 
supervision of principals and focused on other areas such as ADL implemen- 
tation at the high school, and on bringing in other programs districtwide. 
Principals seemed to Interpret this shift of emphasis as freedom to 
continue or discontinue ADL practices as they saw fit. The overall level 
cf implementation* decreased, although there was variation from school to 
school. 
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Technical Assistance to Build Capacity 

The following discussion focuses on technical assistance provided by 
BSC and district staff relative to the classroom variables and the four 
supporting program elements described In chapter 3. Each section describes 
the strategies employed to foster and sustain implementation, and the 
extent to which these strategies were successful. Interview and question- 
naire data related to Implementation of four of these five areas are 
summarized in Table B-2 of Appendix B. Although in many cases strategies 
involved more than one program element, this framework facilitates a 
logical discussion. Also, in most cases the discussions of BSC technical 
assistance are shorter for 1982-83 and 1983-84 than for the field test 
year. During these years, the BSC linker was proactive on fewer issues as 
a result of the district leadership assuming increased responsibility for 
the program* The focus of the linker's assistance continued to be on the 
role of the superintendent, while much less time was spent observing 
principal and teacher performance during these years. 

Attention to classroom variables . As discussed in chapter 2, ADL 
gives special attention to four classroom variables which research has 
shown are highly related to student achievement. They are: (1) prior 
learning; (2) student engaged time; (3) coverage of criterion content; and 
(4) academic performance. The New Jersey district attended to all four 
classroom variables during the three-year period, and related technical 
assistance was provided by both BSC and district staff. The BSC linker 
provided direct support to central office staff, including the superinten- 
dent, the assistant superintendent, and members of the district office's 
Department of Instruction (DOI) through informal and formal meetings, 
training sessions, and seminars. 
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During the field test year, BSC technical assistance was proactive, 
and focused on major implementation decisions. For example, in this dis- 
trict, planning for implementation of content management procedures occurred 
after content training for administrators was completed. This was due to 
the absence of necessary prerequisite materials and the lack of planning 
time. DOI staff were given primary responsibility for developing and 
monitoring the implementation of the articulated curriculum for grades K 
through 12, and were supervised by the assistant superintendent. To carry 
out this mission, the BSC, DOI staff, and the assistant superintendent 
developed a format for instructional planning, called a Quarterly Topic 
Plan. The form was analogous to BSC's Unit Topic Plan just as the district's 
curriculum mapping guide was an adaptation of BSC's School Year Planning 
Guide. It required teachers to record prior learning strengths and weak- 
nesses, dates of instruction, and student success rate, as well as the 
sequence of topics to be introduced on a quarterly, rather than yearly, 
basis. The linker, in seminars with DOI, clarified the need for teacher 
accountability in planning, explained procedures for collecting and 
monitoring content variable data, and responded to district plans and 
concerns. 

The BSC linker later assisted the DOI in planning and developing a 
two-day teacher training session on content management. The training was 
to Introduce the guides, tc demonstrate teachers' vse of the guides in 
planning and instruction, and to explain the principal's role in monitoring 
teacher use of the planning guides. Interview data indicated that planned 
principal and teacher activities related to the planning guides were 
performed by all district staff, i.e., all principals monitored teacher 
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instructional plans and teachers attended to the three content variables. 
However, teachers reported that they were more successful in attending to 
prior learning and coverage than to academic performance. This was reason- 
able, as BSC had not developed formal procedures for assessing success or 
mastery at that time, and the superintendent's press was for content 
coverage. Interview data also indicated that all principals conducted 
classroom observations and all teachers attended to student engaged time. 

Monthly meetings of the BSC linker, the assistant superintendent, and 
DOI staff during the field test year dealt with ways the DOI could help 
principals improve their monitoring of teactfer planning and instruction. 
During these meetings, the BSC linker assisted in planning workshops for 
principals. Topics for the workshops were suggested by DOT experiences in 
schools. For example, the linker and assistant superintendent developed a 
workshop in response to DOI reports of principal and teacher confusion 
about recording and monitoring student daily success. The linker also 
developed simulations and role plays to be used with principals and teachers, 
and presentations on particular aspects of research. One example of a 
research-based presentation was on the Joyce and Showers (1980) model of 
lnservice training. Overall, observations of the BSC linker and feedback 
from the district indicated that BSC assistance supplied to the DOI and DOT 
assistance supplied, in turn, to principals and teachers were key elements 
In facilitating this first year implementation. The significant impact of 
the planning guide on Instructional planning and monitoring, confirmed by 
interview data, suggests that BSC technical assistance in this area was 
effective. 
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During the field test year, the linker had ongoing communication with 
the superintendent, some of which specifically related to the classroom 
variables (also see the following section, principal seminars). In one 
instance, the linker assisted the superintendent in responding to teacher 
resistance to using the planning guides. Teachers objected to the addi- 
tional time needed to prepare instructional plans using the new format. 
The linker planned two responses with the superintendent. First, teachers 
in one building were given permission to share the burden of planning by 
preparing multiple copies of the forms for distribution to teachers at a 
grade level. On reflection, the district agreed that the duplication of 
teacher effort was unnecessary, and assumed responsibility for pre-printing 
the forms. Second, to increase awareness and communication the superinten- 
dent decided i& meet regularly with teacher representatives to clarify the 
new processes and respond to concerns. 

The linker's strategy in terms of lines of communication with the 
district was to deal mainly with central office staff. Direct interaction 
with principals occurred mainly during principal seminars, BSC interviews, 
and school visits for BSC data collection. During the linker's visit to 
schools, concerns were usually raised by principals. The linker would then 
discuss the issue with the superintendent, who would follow-up with a 
response, which in some cases was a topic for a seminar, "^r example, 
during one school visit it became clear that the concept of overlap was not 
understood. The principal informed the linker that teachers would not be 
teaching a test-related content area in mathematics (e.g., fractions) until 
after the testing date. After finding that other principals also had some 
difficulty with the concept of overlap, the linker made a presentation on 
the topic at the next seminar. 
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The major district strategy for providing technical assistance to 
principals was to pair each principal with a DOI staff aeaber, thereby 
moving curriculum supervisors closer to daily activities in schools and 
giving principals a readily accessible resource. Curriculum supervisors 
trained new teachers and conducted workshops, observed and conferenced with 
teachers, and assisted principals with instructional planning and monitoring. 
Approximately two-thirds of the teachers interviewed and all principals 
reported receiving DOI assistance. Only one principal and a few teachers 
reported that this assistance was not helpful. This district strategy of 
pairing principals with DOI staff appears to have been very successful. 

During the second and third years, the linker spent less time providing 
technical assistance related to the classroom variables. Pursuant to BSC's 
strategy of transferring leadership to the district, the linker became less 
proactive, but continued to be responsive to specific concerns and problems. 
Many of these concerns were relayed to the linker through the assistant 
superintendent and the DOI. Examples of such concerns were; what score 
report format best provides prior learning data; and how to standardize the 
definition of student success. The linker also continued to clarify, for 
the superintendent and assistant superintendent, issues and the consequences 
of policy changes, e.g., the number of classroom observations, and the 
necessity of classroom time logs. On occasion, the linker assisted the 

/ 

superintendent by providing him copies of significant research articles and / 
reports. The DOI continued to assist principals and teachers with specific; 

/ 

/ 

issues. 

/ 

« 

The ..linker's assistance was usually successful unless suggestions were 
contrary to district routine. In one case the linker suggested that the 
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district have their California Achievement Test (CAT) data reorganized by 
incoming classes. The district decided that this policy change would not 
be worth the effort, considering the high student turnover. Instead, the 
district asked teachers to use the results of a diagnostic/ prescriptive 
test to assess students' prior learning. After two years, it became clear 
that these test results could not be processed in time for first quarter 
planning. The district then & greed to reorganize the CAT data. 

Defining success and mastery continued to be district problems. 
Interview data from 1983-84 showed disagreement among principals and 
teachers on definitions, the use of planned standards, and awareness of 

i 

district success and mastery goals. As a result of BSC's feedback to the 
superintendent on interview data, he is currently aware of the need to 
restate district policy concerning standards for student mastery, coverage, 
and planning. 

j Interview data from 1982-83 and 1983-84 show that the classroom 
variablet itlnued to receive attention in the New Jersey district. In 
both years, principals reported observing all teachers using the engagement 
rate form for an average of three rounds of observations. All teachers 
Indicated they attended to time management. However, the number of obser- 
vations they reported is suspiciously higl , suggesting that teachers 

confused "scans" with rounds of observations, and ADL with non-ADL observa- 

i 
i 

t^ons. All teachers Interviewed also reported attending to the content 
variables during 1982-83 and 1983-84, and all principals agreed. Teachers 
uied the new planning format, which had coverage built In, for planning 
dijily and weekly lessons within a marking period. However, two principals 
stiated that teachers used the "lesson plan" format rather than the new 
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/ 

/ 

/ 

platting format* Prior learning was analysed from the results of the 6AT 
and Commercial diagnostic/prescriptive test administered at the beginning 
of each school year. Some teachers also used the Individual student skills 
arnjy in determining prior learning* 

Overall, BSC technical assistance strategies facilitated and reinforced 
the district f s attention to the four classroom variables, particularly 
during the field test year. As expected/ the number of requests for linker 
assistance during the second and third years decreased. In addition, 
district strategies of assigning content management activities to the DOI 
and pairing principals with DOI staff appeared to be highly successful, 

9 

Principal seminars . ADL calls for central office staff to conduct 
regularly scheduled leadership seminars for principals. Principal seminars 
were intended as problem-solving and planning sessions that would help 
maintain improvement of Insttuctlon as a priority and improve ancl sustain 
leadership practices. 

Following the district f s agreement to a full-scale implementation in 
the field test year, BSC developed a proposal listing implementation 
activities and a suggested timeline. Seminars were to be 90-minute 
sessions , occurring once or twice a month as part of the district's regular 
principal meetings. 

The BSC linker planned and led all seminars during the field test 
year* The linker selected seminar topics* which generally provided skill 
training for impending implementation events, ^nd met with the superinten- 
dent prior to each session to review the agenda and obtain any recommenda- 
tions and approval. All principals, the superintendent , the assistant 
superintendent, and the assistant to the superintendent attended the 
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seminars. Curriculum supervisors attended the first two seminars but, in 
response to principal objections, were not invited to otter sessions. 

Seminar topics indued: preparing principals to conduct content ami 
time training; participate supervision (conferences and observations); 
coverage; and a review of c* ssroom coding. 

Seminars were also an occasion for principals to express implemen- 
tation concerns. The superintendent often responded to those concerns at 
subsequent seminars. In the interim, the superintendent discussed the 
issues with other central office staff or his administrative council and 
the BSC linker. For example, the principals requested during a seminar 
that the distrir not mandate use of the time logs. The forms were viewed 
as redundant with teachers' weekly lesson plans, and teachers were suspected 
of inflating allocated times in order to increase student engaged time. 
The superintendent decided to enforce the scheduled time minimums but 
re-evaluate the need the time logs. He communicated this decision to 
principals during a subsequent seminar. 

During interviews, all principals reported attending these required 
seminars. The superintendent indicated that principal seminars reflected a 
change in district policy. That is, the view of the principal as instruc- 
tional leader replaced the view of the principal as building manager. The 
assistant superintendent indicated, during informal conversations with the 
linker, that having the linker lead the seminars not only provided needed 
technical assistance to principals, but also gave th<» central office staff 
opportunities to observe principals, to better understand and discuss 
principals' points of view, and to increase their own understanding of 
instructional issues. 
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The proposal for the 1982-83 school year did not include a specific 
timeline for implementation events (including seminars) or BSC's role in 
the seminars . BSC was to assist the central office in developing principal 
leadership upon district request « It was hoped that this strategy would 
foster district independence. The first year of implementation had been 
successful and the district planned to continue the implementation at the 
elementary level and allow the linker to focus on expanding the program to 
the district's high school. 

W 

BSC received three requests for presentations during the 1982-83 
school year, two for principal seminars and one for a 001 seminar. The BSC 
linker conducted a half-day session during administrative training in the 
summer of 1982. The session was a review of the AOL model and procedures, 
and focused on principal leadership functions and procedures for collecting 
and analyzing data* Later in the school year, the assistant superintendent 
lftvtted the BSC^s dirpctor to speak to the DO! on the classroom variables, 
and to give special attention tc coverage and the relationship between time 
and coverage. As a result of an enthusiastic reception from the DOT* the 
superintendent suggested that a similar presentation be given during a 
principal seminar. The BSC director complied with this request- 

It is unclear to what extent principal seminars continued throughout 
the year without BSC leadership. During an interview, the superintendent 
indicated that principals participated in a six-day inservice during the 
summer of 1982, and in monthly principal seminars* Session leaders included 
himself, the assistant superintendent, and consultants; topics were both 
specific to ADL, such as content overlap, snd included other programs such 
as one emphasizing teaching styles and strategies. The superintendent 
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indicated that topics were suggested by state mandates, principal feedback, 
principals' yearly evaluations, and superintendent/principal conferences. 
Principals indicated that they attended monthly seminars but BSC did not 
ask what transpired during seminars. 

The 1983-84 proposal was similar to the previous year's proposal in 
that it included only a general timeline for implementation activities. 
Again, BSC's technical assistance for elementary school principals (includ- 
ing planning and developing seminars) was to be provided upon district 
request. The BSC linker was invited to make three presentations during the 
school year. As part of a seminar held early in tn<» year, the superinten- 
dent: asked the linker to focus on the relationship between the new district 
effort, the teaching styles and strategies program, and the supervisory 
aspects of ADL. Unfortunately, the linker did not have sufficient notice 
to obtain information concerning the new program. Principals and district 
administrators could not provide the needed information either before or 
during the seminar. Consequently, the linker simply reviewed ADL partici- 
patory supervision. 

Late in the fall, the assistant superintendent was concerned that 
principals' implementation levels had dropped and requested a principal 
seminar on participatory supervision. In the seminar, the linker discussed 
classroom observations and learning events using a newly developed time 
management videotape. During this session, it became clear that some 
principals had forgotten some basic skills or concepts, such as the defini- 
tions of engaged and unengaged behaviors. Wide variation in participatory 
supervision procedures was also apparent. As a result, the assistant 
superintendent and the linker planned to conduct a joint seminar on 
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supervision during a specially called principals meeting. The BSC linker 
conducted this seminar without assistance , as the assistant superintendent 
was ill, During the session, the linker and principals worked through a 
case study exercise with sample data. 

J 

Again, BSC was not aware of the extent to which principal seminars 
continued in 1983-84 without BSC leadership. During an interview, the 
superintendent reported that principals were provided with a six-day summer 
session and monthly seminars conducted by central office staff or the 
^linker. He indicated the topics were suggested by the district's end-of- 
year needs assessment and by principal and DO I requests. A little over 
half of the principals indicated that four or five ADL elements (i.e., 
coverage, prior learning, academic performance, time management, partici- 
patory supervision) were major parts of principal meetings, while the 
remaining principals felt only two or three of these issues were covered. 
The frequency or depth of the discussions is unclear. 

Principal seminars during "the field test year would probably have not 
occurred or remained focused on ADL without BSC assistance. The seminars 
were successful for several reasons; they augmented initial, training by 
providing additional time to clarify and discuss issues; they provided an 
opportunity to develop and revise procedures and to identify ami address 
common problems across schools; and, as part of the existing principal 
meetings attended by the superintendent, they indicated the district's 
strong support of the program. The superintendent and assistant superin- 
tendent planned and led most seminars, without linker assistance, during 
the 1982-83 and 1983-84 school years. Although it's difficult to assess 
these sessions, it is clear that principals were aware of a continued but 
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decreased district emphasis on ADL-related issues: leadership, instruc- 
tion , achievement* 

Principal/ r?a cher conferences * As discussed in chapter 3, regularly 
scheduled principal/teacher conferences provide an opportunity for principal 
and teacher to review data on the classroom variables and develop plans for 
action related to Identified improvement needs* BSC suggested that these 
conferences be held following each round of observations * usually three 



times a year* 

, Karly in the field test year, the superintendent and assistant super- 
intendent becaae concerned that principals would not follow through with 
the initial training they had received. They asked BSC for a procedure to 
hold principals accountable for implementing their training in their 
respective schools. In response to district leadership's Initial request, 
BSC invited the superintendent and assistant superintendent to RBS to 
propose a principal/ teacher conference form, and to review the ADL leader- 
ship plan. This initial conference form was a four-page document of 
questions on the status of each of the focus variables. 

During the field test year, BSC technical assistance regarding 
principal/teacher conferences focused on the use of the conference form to 



collect data and select improvement strategies* As mentioned in the 
previous section (principal seminars)* the linker discussed the form during 
principal seminars* and worked with principals using simulated conference 
form data* The linker discovered that most principals lacked the inter- 
personal skills required for effective conferencing* The district printed 
the conference forms and distributed them to principals to use in November/ 
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December -following the first round of classroom observations* 
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Teachers responded negatively to the amount of time required to 
collect and record conference for* data (which claimed part of their 
preparation time), and feared the data were being used in their professional 
evaluations. These concerns were conveyed to the local and state level 

teacher associations. In response, the superintendent requested that the 

e 

linker spend more time in seminars dealing with the principal's role in 
conferences and the intended outcomes. The superintendent also suggested 
that the linker (while* collecting the forms) and the DOI obtain principal 
and teacher feedback to help pinpoint areas of difficulty. The linker and 
DOI found that; principals were having difficulty bringing conferences to 
successful closure, i.e., targeting classroom Improvement strategies; there 
was confusion as to how the conferences differed from the evaluation system 
used to determine teacher tenure and salary increments; and there was wide 
variation in principals' ability to communicate and interpret data with 
teachers. The superintendent responded to these reports during a principal 
seminar by strongly emphasizing the Importance of conferences as part of 
the principal's leadership role and as a formative evaluation of 
instructional improvement and classroom leadership. More specific Issues 
and procedures were addressed by the linker in subsequent seminars. 

Most teachers interviewed during the field test year reported that 
principals observed their classes approximately four times and followed-up 
with jan average of four conferences. The large range in the number of 
conferences (see Table B-2, Appendix B) reported (0-10) suggests that some 
teachers may have confused ADL and non-ADL conferences, or formal and 
informal conierences. Most principals indicated they conducted three 
conferences in accordance with district guidelines. A large majority of 
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principals and teachers indicated that they welcomed the opportunity to 
talk together, on a one-to-one basis, about specific aspects of classroom 
Instruction. 

The following year, 1982-83, although the number of required confer- 
ences did not change (three) , principals were confused as to which version 
of the conference form was official, i.e., the original four-page form, or 
an unofficial shortened version which the linker developed and shared with 
the district during a training session. Although the shorter form was not 
adequate for the intended outcomes of conferences, the district approved 
both forms. The superintendent wanted to comply with principals 1 request 
for a shorter form — a minor concession in return for adherence to the 
larger goal. Linker assistance with conferences was provided to principals 
during administrative training in the summer of 1982, and later in the year 
during a principal seminar. BSC did not collect conference form data 
during the 1982-83 school year. Following the program model, the superin- 
tendent was to monitor principals' conferencing during superintendent/prin- 
cipal conferences. 

During interviews, all principals indicated that they conducted a 
minimum of three conferences with all teachers during the 1982-83 school 
year. The mean number of conferences reported by teachers was four. 
Again, the range in the reported number of conferences suggests that some 
teachers confused ADL and non-ADL conferences, or that some teachers needed 
and were provided with additional support. All teachers said they reviewed 
classroom data and discussed improvement opportunities during conferences. 
Likewise, all principals used one of the versions of the principal/ teacher 
conference form. Thus, although conferences continued as required, the 
quality of conferences is unknown. 
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At the beginning of the 1983-84 school year, BSC officially shortened 
the conference for* from four to two pages. The linker presented this 
revised form to the superintendent who duplicated the form and distributed 
it to principals during a principal seminar. . Again, the linker provided 

e 
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technical assistance related to conferences during three principal seminars. 

During the 1983-84 school year, all principals reported having three 
conferences with all teachers. Although most teachers reported having only 
two conferences, the teacher interviews were administered in March 1984 and 
additional conferences may have been held that year. During interviews, 
all principals, and teachers from all bot one school reported using the 
conference form and discussing all classroom variables. The one exception 
was a principal who ured the engagement rate form and focused <jfc time 
management. Many principals and teachers also reported having Informal 
meetings and discussions concerning the program, classroom data, or instruc- 
tional improvement Issues. As a result of conferences, a little more than 
half of the teachers interviewed felt that they were helped with instructing 
students, exactly half of the ^teachers felt that they were helped.wlth 
managing the classroom, and a little less than half of the teachers inter- 
viewed felt that they were helped with their planning. / 

A major vehicle for participatory supervision by principals is the 
principal/teacher conference. It seems clear that conferences would not 
have occurred without BSC's extensive technical assistance during the firgt 

year of implementation. District requirements and support were also key 
i 

elements in implementing and sustaining conferences. Throughout the 
three-year period, DOI assisted principals with conferencing, both directly 
(i.e., attending or leading conferences) and indirectly (i.e., assisting 
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with J^ta collection or with implementing outcome strategies)* District 
actions reflected central office leadership's belief in principal/teacher 
conferences as a valuable means for monitoring principal implementation. 

Superintendent/principal conferences . The ADL leadership plan calls 
for the superintendent to conduct formal conferences with each principal to 
review the outcomes of conferences with teachers 9 and to discuss improvement- 
opportunities. Through individual principal conferences, the superintendent 
gains perspective on implementation both within and across schools. BSC 
suggests that the superintendent meet with each principal at least two 
times during the school year. 

BSC's proposal for implementation in the New Jersey district for the 
1981-82 school year specified that the superintendent would conduct two 
rounds of superintendent/principal conferences* Although the superintendent 
h#d conducted conferences with principals in the previous year, the focus 
of the conference was new* To assist the superintendent in conducting ADL 
conferences, BSC staff met with the superintendent in October to review the 
purpose of the conferences and introduce the tendent/principal A 

conference formii At that time, the form consisted of two pages of general 
questions summarizing building activities related to implementation. The 
superintendent agreed to use the form* 

Additional technical assistance specifically related to the superin- 
tendent's conferencing was minimal. However* the linker adopted other more 
general strategies which helped the superintendent focus on effective 
principal's behavior. For example, the linker recommended journal articles 
on supervision and school effectiveness research and suggested ways the 
superintendent might help principals deal with their own or their teachers' 
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difficulties. The superintendent presented the conference form to principal 
during a principal seminar, and briefly described the conference procedure 
and a suggested timeline (I.e., aid-year and end-of-year.) 

During this first year of implementation, the superintendent took two 
important steps to incorporate superintendent/principal conferences into 
district routine and to assume diatrict ownership of the conferences . The 
first step was to combine the program's end-of-year conference with the 
state mandated principal evaluation- The principals 9 union t unlike the 
teachers 9 union* did not object to this procedure. Second, the superintend 
dent added a self -evaluation checklist to ADL's Principals Handbook . The 
checklist vas to be completed by each principal prior to the end-of-year 
conference. Although the checklist* did not directly relate to the classroom 
variables, it was an attempt by the superintendent to assume ownership of 
RSC-deve loped materials. 

As planned t two conferences were conducted with each of the nine 
elementary school principals. Principals reported discussing all four 
classroom variables, although Interviews did not ask for an assessment of 
conferences. The superintendent reported that the conferences were bene- 

flclal both in helping him focus his ova Ideas on Instruction and In 

A 

assuring that prlnolpals vere constantly avare of the instructional issues 

9 

addressed by the improvement program. Informally, the superintendent 
indicated to the linker that conferences also provided opportunity to 
follow up on principal and teacher problems, such as earlier teacher 
complaints about their participation in conferences. Although be did use 
the conference form, the superintendent Indicated to the linker on several 
occasions that the questions on the form were too general to facilitate 
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discussion of the classroom variables. For example* in some cases princi- 
pals, in an effort to be positive, described their buildings as "problem 
- free," thus cutting off any opportunity for discussion. 

The following year, 1982-83, the sain technical assistance provided by 
the linker vas the revision of the conference fora. The new form, presented 
to the super intendene in November 1982, asked about specific levels of each 
classroom variable to be summarized across teachers. In order to secure 
principal and teacher approval for the revised conference form, the super- 
intendent modified the form so that It represented grade levels, rather 
than individual^classrooms. The fora was distributed to principals by the 
superintendent during a principal seminar. The superintendent also gave 
principals a checklist of materials needed for the conference. 

During an interview, the superintendent reported conducting at least 
two conferences with all principals in 1982-83, and in two cases, two 
follow-up conferences were needed and conducted. He reported using the 
conference form to discuss the four classroom variables for reading and 
mathematics, along with principal's conferences with teachers. He reviewed 
individual conference forms and improvement plans. The majority of prin- 
cipals reported having two conferences, one principal reported four, and 
one reported having one conference. All principals completed the school 
summary section of the conference form although three principals reported 
dealing mainly with student engaged time. Only two principals indicated 
conferences wer«* helpful; one principal felt they helped with training and 
supervising teachers and the other principal indicated planning with and 
supervising teachers as areas of benefit. 
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During the 1983-84 school year, the district continued to require two 
superintendent/principal conferences and use the district's modification of 
BSC 1 a conference form. Although the end-of-year conference continued to be 
tied to the principals 9 evaluation, this year the earlier conference was 
tied to the state's spring Monitoring of basic skills achievement and other 
indicators of district ef fectiveners. During principal interviews in May 
1984 f > «K>st principals reported having had only one conference with the 
superintendent , although another was planned. One principal reported 
having two, and two principals reported having three conferences* One 
principal did not use the conference form* However, all principals discussed 
the four classroom variables during the conference. Although only three 
principals felt that conferences provided specific help with their leader- 
ship functions, most principals Indicated that the conferences demonstrated 
the superintendent's commitment to the program and his concern and support 
for principal performance* 

Throughout the three-year period, the superintendent conducted princi- 
pal conferences in accordance with BSC's proposal* BSC's technical assist- 
ance in developing the conference form provided the needed structure for 
the conferences. Although conferences provided an opportunity for the 
superintendent to monitor principals' effortt in helping teachers to 
improve instruction, they seemed to also serve a more important function; 
that is f the demonstration of superintendent interest and commitment - This 
commitment was also reflected in the superintendent's actions to incorporate 
conferences into, district routines and to adapt materials to better meet 
district needs. However, it is possible that the modification of the 
conference form which the superintendent perceived as necessary for 
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principal and teacher approval (i.e., data summary by grade level) marked 
declines in levels of achievement and classroom process variables in 
Individual classrooms. 

Differentiated inservice . One of ADL's overall goals is to increase 
the district's capacity for improvement and reform. Differentiated 
Inservice is a means to achieve that goal. The leadership plan suggests 
that the central office provide small or large group inservice based on 
needs identified in conferences, seminars, and informal discussions. 

The superintendent in the New Jersey district supported principal 
inservice as a weena to promote and support instructional improvement. 
Prior to the fie\d rest year, the central office had scheduled, principal 
inservice sessions which were developed and presented by non-district 
consultants. During the field test year, these sessions were principal 
seminars and focused on ADL activities. 

In addition to inservice provided by the linker to all principals 
during principal seminars, DO I staff were assigned to each school to 
support principals by helping with school level problem solving, and to 
insure that implementation was proceeding as planned. Although the extent 
of DO I involvement varied across schools, most principals and central 
office staff interviewed at the end of th< tield test year agreed that 
central office otaff spent more time that year supporting basic skills 
instruction and were more effective in their efforts than in previous 
years. On occasion the linker s^so provided individual technical assis- 
tance to principals at the superintendent's request or with his approval. 
For example, one principal asked the linker to review instructional plans 
which teachers had developed for classes which contained two grade levels. 



9 

ERIC 



109 

108 



In other Instances, principals asked the linker to examine completed 
conference forms. The potential difficulty with the strategy of having the 
linker provide individual principal assistance is that it may foster 
dependence rather than building capacity. 

In the field test year* the district also used in service to respond to 
the needs of the district f s lower achieving schools* In mid-year , the 
district asked a local educational agency to conduct a series of teacher 
workshops on Instructional techniques for presenting reading and math 
content. The agency proposed to work with only a few schools in the 
district. Although the idea of differentiation was new to the district, 
the assistant superintendent agreed tp the agency 9 s proposal and selected 
the four lower-achieving schools for participation. The BSC linker assisted 
in planning and conducting the workshops. The workshop topics were selected 
by principals on the basis of their teacher conferences and classroom 
observations. Principals were also responsible for assigning teachers to 
specific workshops. Each workshop extended over five sessions from January 
to May. The linker led two classroom management sessions tor selected 
teachers. Although^ the district found the workshops to be generally 
beneficial, the logistics of a differentiated inservice were difficult to 

o 

manage (e#£. f releasing selected teachers, locating meeting rooms). 

Ir 1982-83, the district continued to have separate monthly seminars 
for principals and DOI, Topics approved by the superintendent related to 
state mandates and district con cernai along with areas of special interest 
to principals. The assistant superintendent made a separate agenda for 1)01 
meetings. The DOI /principal pairings continued. DOI provided individual 
assistance to principals and inservice sessions to teachers in their 
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content areas. Most principals indicated during interviews that the DOI, 
in additioa to assisting principals with teacher training and participatory 
supervision, either trained, or would train, new teachers in the district's 
Instructional procedures. 

For the 1982-83 and 1983-84 school years, a few principals reported 
receiving DOI assistance in areas of need identified during superintendent/ 
district conferences. However, nost principals felt that the conferences 
were conducted aore for reinforcing the program and principals' accounta- 
bility than for identifying and responding to principal needs. During the 
1982-83 principal conferences, the superintendent reported that no needs 
which related to principal leadership functions were identified. The fol- 
lowing year he reported that principals' needs for help with planning and 
monitoring were identified during conferences. The superintendent indicated 
that he either assisted these principals individually, or referred them to 
the assistant superintendent, to other principals, or to training tapes. 

During 1983-84, district policy and procedures regarding Inservice 
remained unchanged. Inservice continued in the form of whole group sessions 
at various levels, along with individual DOI assifftanc* to principals and 
teachers. However, there were indications that the district saw different- 
iated inservice as a viable strategy. One example was a DOI inservice 
series on Instructional techniques in mathematics for all seventh and 
eighth grade teachers. The decision reconduct the inservice was based on 
an analysis of the previous year 'attest results and the district's concern 
with ninth grade student performance on the state mandated achievement 
test. The DOI developed training and practice materials for teachers. DOI 
again conducted workshops for tjeachers new to the district. 
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As a result of 1982-83 and 1983-84 principal/teac'ner conferences, many 
teachers reported receiving assistance with planning, c laser oca management, 
and Instruction. Although in the majority of cases this assistance was 
provided on an individual basis (probably during principal/teacher 
conferences and classroom observations) , teachers reported being provided 
with inservlce and materials as an outcome of conferences* 

During the three-year period, the district seems to have been respon- 
sive to individual and group needs. However, ' *e district was and still is 
reluctant to support different iatlou as a grading principle for inservlce. 
The district strategy was to provide for total group inservlce, differen- 
tiated by level and group need, and to pair individual principals and DOI 
to meet more specific school needs. There was some differentiated inservlce 
during the three-year period but this raised planning problems (e.g., 
^logistics), and the district was reluctant to single out staff with selected' 



> needs. Although DOI assistance at the school level is hifehly dependent on 
the principal /DO I relationship, and on individual competence and interest, 
the district feels that these practices have been successful and should 
cont inue . 

Technical Assistance for Evaluation 

One of BSC's overall strategies was to provide technical assistance 
for evaluation as a jpeana of maintaining districts 1 focus on the goals of 
improved instruction and increased student achievement* During interviews, 
BSC assessed participants* perception of changes in their and others' roles 



as Instructional leaders. Although limitations on BSC resources prevented 
the systematic collection and analysis of data on the classroom process 
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variables (i.e., critical student behaviors), student achievement data were 
analysed by both the district and BSC during the three-year period. This 
section begins with a discussion of district staff* s perceived changes in 
role, organized by level (superintendent, central office staff, principals, 
teachers). It is followed by an analysis of student achievement data 
during the period 1980-1984. Both sections include reviews of BSC and 
district strategies used to facilitate and carry out formative evaluation. 

Role changes . A major goal of ADL is to have district staff adopt the 
role-related functions prescribed in the leadership plan. BSC' a intent is 
for district staff to change their behavior in accordance with those 
functions specified in the plan. The information on role changes for 
educators in this district is baaed primarily on interviews administered to 
the superintendent, *.ll principals, and 18 elementary school teachers. The 
data were collected during the 1983-84 school year (see Appendix A for the 
interview schedule) and deal with the period 1981-84. Data collected 
during the field test year are also presented where appropriate. 

In interviews the superintendent stated that his role as instructional 
leader had changed somewhat as a result of the instructional improvement 
program. He thought the program had formalized the supervision process and 
focused his own leadership behaviors. In contrast, the majority of the 
principals noted in their interviews that the superintendent's role did not 
seem to change. In most of these cases, principals felt they had no 
standard for comparison because the program was implemented districtwide 
soon after the superintendent was appointed. 

On end of the field test year questionnaires* most principals and _»any 
teachers indicated that central office staff, in general, spent more time 
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In 1981-82 planning and supporting basic skills instruction. Several 
principals stated in end~of year interviews that central office curriculum 
supervisors (DOI) helped then by observing classrooms, holding teacher 
conferences, and assisting with teacher training. Principal and teacher 
interviews conducted in 1983 and 1984 confirmed that the DOI continued this 
assistance during these two years. 

By 1984, most principals thought their own leadership role had changed 
a great deal. The superintendent and many of the teachers interviewed 
agreed with this assessment. The changes most commonly mentioned by 
principals were more frequent classroom observations and teacher confer- 
ences, and improved monitoring of teacher coverage of instructional material, 
i .1 ■ 

; i 

Most teachers interviewed felt that their own role as instructional 
leaders had changed a great deal or somewhat. The superintendent stated 
that the teachers' role had changed a great deal as a result of the program, 
whereas principals felt that the teachers' role had changed somewhat. 
Although some of the teachers interviewed thought the changes were negative 
(e.g., more paperwork, less freedom, less creativity), a larger proportion 
mentioned positive changes, such as improved methods of instructional 
planning and classroom management. 

The BSC linker assisted in the district's evaluation process by 
providing the district with BSC's analyses of district data and developing 
formal proposals based on these analyses. As mentioned earlier, each year 
the linker developed a formal proposal outlining district implementation 
activities and BSC technical assistance activities for the school year. 
The proposals were based on test results, interview data, conversations 
with the superintendent and assistant superintendent about district goals, 
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and principal feedback during seminars. As a model, the first proposal 
contained very specific deadlines/ timelines which the district approved and 
followed. Later proposals were less specific as the linker felt the 
district would assume this planning responsibility based on previous 

experience. However, district leadership tended to focus on positive 

* . \ 

outcomes rather than on areas in need of improvement , i.e., the district 

N 

did not use implemen tat ion data on leadership roles and classroom process \ 
variables in planning for the 1982-83 and 1983-84 school years. Principals 
were expected to follow the 1981-82 timeline. But, without directed 
guidance in the form of new or revised memos or timelines, principals had 
difficulty remembering the cycle of activities. 

During the field test year, data on classroom process, variables were 
collected and analysed by BSC, and provided to the district in a summary 
report. However, these data were not formally used by the district or 
prepared during the following two years. Informally, the superintendent 
gathered information about classroom processes through meetings with 
principals, the BSC linker, and DO I staff, and used this information in 
program planning. For example, based on these discussions, the superin- 
tendent felt that AOL was successful in providing a basic foundation for 
Improving a low achieving district; However, as a result of an additional 
perceived need to help teachers raise expectations and enhance student 
potential, another district effort which emphasized teaching styles and , 
strategies was adopted. The superintendent was concerned that adherence io 
the curriculum without a concommltant change in teaching techniques woujd, 
iu the long run, have depressed students' potential for achievement. / 
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At Che end of each school year* the BSC linker also prepared a summary 
of the year's interview results and developed some suggestions for the 

district's future planning * The summaries were shared with the superintend 

! . j ; 

dent and assistant superintendent, In some cases 9 the linker's suggestions 
were followed; in other cases, the superintendent decided action was not 
warranted or considered several alternatives* The data summary^SC 
v prepared at the end of the 1983-84 school year was presented to principals t 
in a more general form, during a leadership seminar. This clarified for 
principals the need to Increase ADL monitoring and Implementation to 
improve instructional leadership and achievement. During leadership 
seminars and informal meetings, the BSC linker also encouraged district 
staff to use information from conferences, discussions, and observations 
for planning . For example, the linker suggested that the superintendent 
use this process to develop seminar topics and additional coaching for 
principals. 

As previously mentioned t the district's major strategy for formative 
evaluation of instructional leadership and classroom process variables 
the informal collection and evaluation of information. Principals also 
conducted informal evaluations for planning purposes. At the end of each 
school year, principals were required to develop a school year plan for the 
following year, based on a review of available data. The plans were 
submitted to the superintendent for approval. However 9 it is the linker's 
impression that the plans were general and did not refer to specific 
classroom data collected through use of the improvement cycle. 

Student achievement . The New Jersey district administers the California 
Achievement Test (CAT) to all students in the spring of each year. Scores 
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* for grades 1-8 for 1980-84 are presented in Table 2 as NCEs. The table 
also presents gains from 1980-81, 1981-82, 1982-83, and 1983-8*. (See 
Appendix C for a discussion of the significance of achievement gains.) The 
scores in the horizontal rows represent different groups of students in 
each successive year. 

Before the 1981-82 field test, low student achievement was a cause for 
district concern. Only one grade out of eight scored above the national 
average (50 NCEs) in reading; six grades out of eight scored above the 
national average (50 NCEs) in math. Average gains froa 1980-1981 were" only 
+1.4 for reading and +1.5 for math. The newly appointed superintendent 
felt that ADL had significant potential for improving basic skills 
instruction and achievement, and decided to implement the approach in 
grades K-8 districtwide during the 1981-82 school year. As Table 2 
illustrates, students performed markedly better in the spring of 1982. Two 
grades in reading (grades 1 and 2) and seven grades in math (grades 1-7) 
scored above the national average of 50 NCEs. All eight grades 
demonstrated gains in both reading and math. Changes in reading averaged 
+4.0 NCE points and ranged from +1.3 to +7.9 NCEs. Changes in math 
averaged +5.1 NCEs and ranged from +2.3 to +12.9 NCEs. - 

Blester et al. (1983) present m analysis of these test data by 
school. This analysis suggests a relationship between level of ADL imple- 
mentation and student achievement in reading and math. Achievement gains 
were most positive for those schools with the highest levels of imple- 
mentation. The analysis also revealed a link between length of involvement 
and gains in achievement ; two of the three schools that implemented 
program elements prior to the field test implemented the program more fully 
in 1981-82 and showed the greatest gain in student achievement. 
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Table 2 



Student Achievement Scores*: 1980-34 
New Jersey School District 

READING 



f?T*Jlff ft* 


, — 

Scores 


Gains/Losses 


1980 


1981 


1982 b 


19C3 


1984 


80-81 


81-82 


82-83 


83-84 


1 


50.2 


54.6 


55.9 


56.6 


54.9 


+4.4 


+1.3 


L 

+0.7 


-1.7 


2 


46.3 


47.5 


51.5 


51.5 


55.5 


+1.2 


+4.0 


0 


+4.0 


3 


47.1 


44.4 


48.5 


52.0 


50.7 


-2.7 


+4.1 


+3.5 


-1.3 


4 


43.2 


44.2 


48.7 


48.4 


49.4 


+1.0 


+4.5 


+0.3 


+1.0 


5 


45.2 


45.7 


48.7 


50.1 


50.4 


+0.5 


+3.0 


+1.4 


+0.3 


6 


41.8 


42.8 


45,9 


49.2 


50.3 


+1.0 


+3.1 


+3.3 


+1.1 


7 


36.0 


39.4 


47.3 


48.2 


49.9 


+3.4 


+7.9 


+0.9 


+1.7 


8 


36.0 


38.3 


41.9 


49.5 


50.4 


+2.3 


+3.6 


+7.6 


+0.9 


Mean 


43.2 


44/6 


48.6 


50.7 


51 ..4 


+1.4 


+4.0 


+2.1 


+0.7 



MATHEMATICS 



Grade 


Scores 3 


Gains/Losses 


1980 


1981 


1982 b 


1983 


1984 


80-81 


81-82 


82-83 


83-84 


1 


54.9 


56.8 


60.3 


60.6 


59.2 


+1.9 


+3.5 


i 

+0.3 


-1.4 


2 


50.7 


52.4 


57.5 


56.8 


59.1 


+1.7 


+5.1 


-0.7 


+2.3 


3 


56.4 


52.6 


56.5 


59.1 


58.3 


-3.8 


+3.9 


+2.6 


-0.8 


4 


52.4 


51.3 


57.8 


57.7 


56.0 


-1.1 


+6.5 


-0.1 


-1.7 


5 


53.7 


55.4 


58.4 


60.6 


62.3 


+1.7 


+3.0 


+2.2 


+1.7 


6 


50.4 


55.4 


57.7 


61.2 


61.0 


+5.0 


+2,3 


+3.5 


-0.2 


7 


41.8 


43.5 


56.4 


59.5 


60.3 


+1.7 


•■12.9 


+3.1 


+0.8 


8 


38.1 


42.7 


46.3 


54.5 


57.2 


+4.6 


+3.6 


+8.2 


+2.7 


Mean 


49,l< 


51.3 


56.4 


58.8 


59.2 

0 


+1.5 


+5.1 


+2.4 


+0.4 



Scores represent the results of California Achievement Tests (CAT) as 
Normal Curve Equivalents (NCEs). 

^ADL was first implemented on a districtwide basis in 1981-82. 
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The upward trend in achievement gains continued over 1982-83, although 
the Magnitude of the gains decreased. Four grades in reading and all eight 
In math scored above the national average. Seven of eight grades in 
reading and six of eight grades in math registered increases. The average 
change in reading was +2.1 NCEa, with changes ranging froa 0 to +7.6 NCEs. 

4 

Changes In math averaged +2.4 NCEs, and ranged from -0.7 to +8,2 NCEs. 

In 1983-84 , the positive trend was still apparent but again to a 

lesser degree. Six grades in reading and all eight grades in math scored 

« 

above the national average of SO NCEs* Six grades in reading and four 
grades in math demonstrated gains. The average change iii reading decreased 
to +0.7 NCEs, with changes ranging from -1.7 to +4.0 NCEs. The average 
change in math was +0.4 NCEs, with a. range from -1.7 to +2.7 NCEs. The 
superintendent attributed the decrease to an overall lessening in the 
degree of implementation sparked by his own relaxation of supervision 
procedures* and his initiation of new district efforts , such as the teaching 
styles and strategies program. In addition, the large amount of time the 
superintendent devoted to sharing the district's success with outside 
educators (in conferences, symposia, etc.) detracted from the available 
time and perceived need to stabilize and institutionalize the new practices. 
However, in spite of the decrease in the magnitude of achievement gains 
over the last two years, the substantial initial increases were maintained. 

In the New Jersey district, central office staff examined achievement 
data at the end of each school year to determine how grades within a school 
were distributed about the mean* The reports, which Included a ranking of 
schools, were presented to the school board, and each principal was provided 
with a summary for his or her building. In addition, during the 1982-83 
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and 1983-84 school years, DOI staff analyzed achievement scores by objective. 
A sumary of the resulting strengths and weaknesses was provided to each 
principal for incorporation into school year improvement plans* 

To supplement these district summaries » each year BSC analysed the 
amount of gains/losses for each grade within a school, and provided other 
data upon occasion (e # g*. comparisons of several years 9 data at the school 
or grade level, the relationship between school level implementation and 
achievement). The superintendent used some of this information In program 
planning. For example* as a result of the dramatic increases in student 
achievement during the field teat year, the superintendent reduced the 
number of required classroom observations and the number of principal 
seminars led by BSC* Then, when achievement scores suggested a declining 
trend during the 1983-84 school year, plans were developed to Increase 
observations , BSC involvement, and the superintendent 9 s monitor Inge 

Technical Assistance and Institutionalization 

/ The long-term goal of institutionalizing improvement practices that 
are based on research findings on effective classrooms and leadership can 
be fscilitated by assisting and encouraging districts to build ADL into 
district routines. In the New Jersey district both BSC and district 
strategies have fostered institutionalization* Some of these strategies 
have been referred to in earlier sections of the report. 

The linker pursued several strategies to sustain implementation and 
foster institutionalization in the New Jersey district. One overall 
strategy. In accordance with the program model and top-down strategy, was 
for the linker to be highly proactive during the field test year, and in 
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subsequent years act as an ongoing resource for the central office in 
providing technical assistance as requested. ^ It was hoped that the district 
would become more independent as the need for external support gradually 
decreased. Over the three-year period, the amount of support, in terms of 
formal and informal meetings with administrators and requests for technical 
assistance, did decrease. Following the field test year, the linker 
communicated more through telephone conversations and letters in order to 
help improve the quality, or "f ine-tune" practices begun in 1981-82. At 
times, linker assistance in problem solving was requested. For example, 
during the 1983-84 school year, the linker was asked to participate in 
planning and seminars related to principals* leadership functions, e.g., 
participatory supervision. 

Another linker strategy, which the research suggests is important to 
institutionalization, was to encourage the integration of ADL forms into 
district activities. The linker modified the forms based on BSC experiences 
and input from district staff. The principal/ teacher conference form was 
revised at the end of each of the three years, and the quarterly topic 
plan, engagement rate form, and superintendent/principal conference form 
were each revised once. Feedback from the district indicated that, in all 
cases, the forms were used and the revisions were beneficial. Although the 
quarterly topic plan did become a districtwide format for Instructional 
planning, the linker was not successful in having the superintendent 
replace or integrate district evaluation forms with ADL conference forms. 

The linker was less successful with two other strategies. The first 
strategy was to have the district adopt the differentiated inservice model. 
Although there was some differentiated inservice during the three-year 
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period, the superintendent was reluctant to fully support the practice* 
The second strategy was to promote formative evaluation. Unfortunately* 
the superintendent did not follow the linker's model for developing yearly 
proposals* i.e.* using implementation data on leadership roles and classroom 
process variables in planning for the next year's implementation. Although 
he did review data, he preferred to trust informal reports from principals. 

The New Jersey district adopted many strategies or supports which 
proaoted institutionalization. One overall support, which according to 
Miles (1983) is the basis for the successful institutionalization of an 
innovation, was the high level of administrative comaltment to program 
practices* particularly from the superintendent „ The superintendent viewed 
AOL as a way to accomplish his goals of improving principals 9 instructional 
leadership (i.e*, getting principals to observe in classrooms) and improving 
instruction and achievement, rather than as an RBS program* This resulted 
in four types of district strategies. 

The first strategy was the inclusion of ADL in initial policy decisions. 
The decision to implement coincided with the major district policy and 
procedure decisions of the newly appointed superintendent* The superinten- 
dent and AOL had common goals* Also, the superintendent saw his role as 
being active, highly visible, and directly supportive of principals* As a 
result, BSC's input in these early decisions was welcomed, particularly in 
areas such as instructional planning and monitoring, and participatory 
supervision* More important, the district assumed ownership of these early 
ADL-related policies and procedures* Indeed, several staff .interviewed 
were not aware that some district procedures had originated from ADL, 
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Second, this administrative coe»lt*ent was reflected in district 
documents. For example, at the beginning of each school year, the superin- 
tendent developed a written mission statement and an action plan which were 
distributed to all district staff* Many objectives in the plan related to 
ADL issues, such as instructional planning and classroom management . The 
superintendent, in pursuing his view of principals as instructional leaders, 
scheduled a two-week staff development program for principals each summer 
during which the action plan objectives for the school year were dealt with 
in iso re detail. This was also an opportunity to support principal inservice 
with organizational change , 

In addition to mandating ADL practices, ADL was built into exls ting- 
district requirements* For example, during the superintendent's supervisory 
conferences with principals a number of other issues, including state ami 

local mandates for which principals are held* accountable, were reviewed. 

* 

In another case, the supe'rihtendent redefined the planning requirement in 
teachers 1 contracts to include the new Instructional planning form t, When, 
teachers documented the extra paperwork Involved, the district responded by 

0 

pre-printlng the elementary level plans as much as possible. Many teachers 
indicated that the plans were the most helpful part of the district program. 

The superintendent also agreed to make prior learning datd available 
to teachers so that the data could be used In instructional planning* At 
first he felt that the cost o£ reorganizing the California Achievement Test 
data by incoming classes was unwarranted in such a highly mobile district. 
As an alternative, the superintendent rescheduled the date for administering 
a diagnostic prescriptive test to early in the school year so that test 
results would be available in time to be used in instructional planning. 
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However, these test data were not provided in tine to be useful for first 
quarter planning. Therefore, during the third year of implementation, the 
district provided teachers with reorganized California Achievement Test 
data. 

Another strategy which the district adopted to foster institutional- 
ization was to Integrate new district programs with ADL. The superintendent 
assumed that after one year of implementation, AOL was a solid foundation 
upon which other programs could b* built. One innovation which was adapted 
was a model for varying teacttlmg styles to match various student learning 
styles. However, this strategy was mot successful for several reasons: 
the two programs were based on different implementation models (top-down 
vs. bottom-up), there was no opportunity for communication or coordination 
between the program linkers, and more importantly, many principals inter- 
preted the new effort as replacing rather than supplementing AOL as the 
focus of principal seminars and superintendent communications shifted. 
Although the superintendent's goal was to continually upgrade the district, 
. he -was overconfident in his assumption about the speed and conditions under 
which institutionalization of an Innovation occurs. 

Fifth, the district developed new policies which supported ADL, either 
directly or Indirectly. One very successful strategy was pairing a DOI 
staff member with each principal lo assist in planning and implementing ADL 
at the school and classroom levels. ;Both DOI staff and principals were 
required to submit monthly Reports on their activities to the superintendent. 
This DOI responsibility begap, during the field test year, and throughout 
the three-year period the pairings continued, although the actual arrange- 
ments of shared* wonk' load varied across schools. During 1983-84 interviews, 
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school-level planning, and, Individual teacher assistance >-JJ*a«ples of 



so st principals indicated that DOI staff assisted with time -on- task observa- 
tions. In soae cases DOI assistance also Included conferences, workshops, 

other new district policies which indirectly related to ADL were: the 
annual ranking of schools based on achievement scores; presenting these 
data to the school board; and adopting a grading and promotion policy for 
students in grades K-8 which supported prior learning, coverage 9 and 
academic performance. Although the latter policy was beneficial in stan- 
dardizing grade policy across the district, it conflicted with the accepted 
procedure of automatically promoting "problem* 1 children. 

One promising sign of institutionalization in this district is the 
spread of ADL elements to new classes and grade levels. In 1982-83, 
special education supervisors began to conduct observations of student 
engaged time. Also, based on the program* s success at the elementary 
level, the district decided to extend curriculum alignment and leadership 
practices to the high school* Planning for high school implementation 
began in the summer of 1982, and has proceeded in accordance *lth district 
and BSC expectations. 

Institutionalization in the New Jersey district has been promoted by 
several factors, the most importan 4 - being that the superintendent was 
committed to th program and mandated dlstrictwide implementation. The 
superintendent provided and arranged for support and assistance with 
implementation; there was stability in program leadership and staff; and 
attempts were made to build program policies and procedures into district 
documents and routines. The significance of district commitment and 
Interest is exemplified in the overall decrease in the level of ADL 
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implementation (and student achievement) during the 1983-84 school year, 
when the superintendent was less active and visible in his support of AOL. 
This change in the superintendent's behavior resulted fro* his assumption 
that ADL would automatically continue. Ironically, during this school 
year, a large portion of the superintendent's time was spent sharing the 
district's success with outside educators. It can be assumed that the 
administration's more visible expression of commitment to ADL in 1984-85 
will again result in a high level of institutionalization. 

In the New Jersey district, ADL practices are embedded in the district's 
organization, procedures* *nd routines* However, variation across years 
suggest that beyond the paper documentation and altered routines, institu- 
tionalization requires, for same unspecified amount of time, active district 
leadership, external support at critical junctures ,\md continued monitoring. 

The Delaware District 

The Delaware district is a small rural district serving a population 
of predominately low socio-economic status. As of 1981-82 there were 
approximately 2,100 students in one kindergarten school, two elementary 
schools (grades K-5), one middle school (grades 6-8), and one high school 
(grades 9-12). Minority group students made up about 20 percent of the 
population. In the fall of 1978, before the district began to work with 
RBS, students were scoring around the national average, but well below the 
state average, on the standardized achievement test administered statewide. 
In fact, the district was among the lowest scoring districts in the state. 
This fact was publicized in state department reports and created some 
public press for the improvement of achievement. 
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During the 1978-79 school year, the district began to cooperate with 



the BSC as a development district. Subsequent program implementation 
occurred over three and one-half years. This cooperative development 
involved principals and teachers from all schools, but participation was 
greatest at a Basics Plus elementary school. During 1979-82, the district's 
central office provided continued support for the program. Top administra- 
tors who played major roles in implementing and maintaining ADL were the 
superintendent, the deputy superintendent, and the director of special 
programs. Although the director of special programs was given significant 
responsibility for overseeing day-to-day Implementation, the position did 
not have line authority over principals. The departure of the district's 
superintendent and the Basics Plus elementary school principal in June 1983 
was accompanied by a reduced emphasis on both the improvement approach and 
BSC involvement. In January 1984, the director of special programs also 
left the district. 

Data from the Delaware district are included in this report for the 
contrast they provide with data from the Pennsylvania and New Jersey 
districts where there were more active district efforts to maintain Achieve- 
ment Directed Leadership and a much greater level of ongoing support from 
BSC. 

Technical Assistance to Build Capacity 

The following discussion is organised around the program's central 
element, attention to classroom variables, and the four program elements 
described in chapter 3. Each section describes the strategies BSC employed 
to promote and sustain implementation of the element and the extent to 
which the element was implemented. The amount of information included in 
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these sections is limited by the nature of BSC's involvement in the district. 
Table B-3 in Appendix B suanarizes Interview and questionnaire data relevant 
to the extent to which the district attended to classroom variables and 
implemented the three elements: principal seminars, principal/teacher 
conferences, and superintendent/principal conferences. 

Attention to classroom variables . ADL gives special attention to four 
classroom variables which research has shown are highly related to student 
achievement: prior learning, student engaged time, coverage of criterion 
content, and academic performance. Much of the formal training related to 
these variables was provided during the period of cooperative prog ran 
development (1978-81). Furthermore, by 1981 the district had already 
completed many prerequisite tasks necessary for content management. For 
example, the district developed reading/ language arts and math curricula 
and selected standard texts during the first year of program development. 
This work was undergirded by the BSC logic and rationale for content 
management; a major criterion in the curriculum development and text 
selection process was assurance that there would be a high percent of 
content overlap with the standardized achievement test. 

For the field test year, BSC linkers provided a good deal of technical 
assistance related to the critical classroom variables. Much of tnis 
assistance involved meeting with the director of special programs and other 
district office staff to review the status of implementation and plan for 
further implementation (e.g., schedule implementation events, arrange for 
appropriate test data) . BSC staff also led and assisted with training for 
administrators and teachers. In connection with these workshops, the BSC 
linker often developed materials later used in implementation. For example, 
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for Che content management workshops In reading/language arts and math, the 
linker developed foras which Indicated the overlap in cjbntent between tile 
year-end test and the district's list of minimum competencies or the tables 
of contents in the district's textbooks. j 

The district also provided technical assistance tfelated to the class- 
variables, often in connection with the teacher Workshops described 
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above. For example, the Title I director analysed th^ spring 1981 stan- 

/ | 
dardized achievement test results to determine levels -of student prior 

learning. Teachers used this information in the content workshops to 

determine student strengths and weaknesses which they ^recorded on the 

overlap forms. 

Interview data indicate that, during the field tes^ year, two rounds 

\ 

of engaged time observations were conducted by the Basic\i Plus elementary 

\ 

school principal. The assistant principal at the other elementary school 
conducted one round of observations for all teachers, and a second round 
upon teacher request and for special education teachers. Central office 
staff and principals Indicated that attention was being given to all three 
content variables — prior learning, coverage, and academic performance. 
Teachers confirmed this in year-end questionnaires. Overall, BSC's 
technical assistance appeared to sustain the district '.s implementation of 
ADL's time and content components during the field test year. 

Prior to the 1982-33 school year, 'BSC and the superintendent agreed 
that- future work would be on a more limited basis. Specifically, the BSC \ 
linker would respond to district requests for assistance, rather than 
assume a proactive role. During the year, the BSC linker helped district 
staff plan and lead a teacher workshop on content management. Interview 
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data indicate that teachers considered not only prior learning but also 
content overlap in planning instruction. In terms of student engaged time, 
the degree of implementation decreased. Only the principal of the Basics 
Plus elementary school conducted classroom observations during the 1982-83 
school year. The assistant principal of the other elementary school, who 
had previously conducted' observations, left the position in June 1982. 

The following year, 1983-S4, BSC technical assistance was not requested. 
As mentioned earlier, the superintendent and the Basics Plus principal left 
the district in June 1983. Prior to leaving the district in February of 
that year, the director of special programs, along with the assistant 
principal at the middle school, conducted a teacher workshop focusing on 
student prior learning. Teachers and principals reported attending to 
prior learning, which suggests the workshop was successful. During the 
1983-84 school year, principals did not conduct classroom observations. 
However, teachers in one school indicated that they were aware of time-on- 
task and that they periodically scanned their own classrooms. The Basics 
Plus principal who left the district in June 1983 continues to conduct 
time-on-task observations in her new position as principal of an elementary 
school in, a large urban district. 

Principal seminars . ADL calls for central office staff to conduct 
regular principal seminars. Seminars are intended as problem-solving and 
planning forums. Their purpose is to maintain instructional improvement as 
a high district priority and sustain and improve leadership practices. 

Central office staff, primarily the director of special programs and 
the deputy superintendent, requested technical assistance with planning and 
leading principal seminars approximately five times during the field test 
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year. Most central office staff and all principals and assistant principals 
attended the seminars. Topics included: an overview of ADL and imputa- 
tion plans; conferencing procedures; a review of standardized achievement 
test results; a comparison of ADL and mastery learning; and a preliminary 
review of field test results. Although initial sessions were led by BSC 
linkers, later sessions were co-led and led by central office staff. BSC 
felt that the sessions were excellent learning experiences and were well 
received. The effect on ADL implementation appeared to be positive. 

During the 1982-83 school year, the superintendent included inservice 
training for principals as part of regularly scheduled principal meetings. 
These sessions, led by district' of f ice staff and/or consultants, were 
designed to keep principals up-to-date on current theoretical developments. 
As BSC was not involved in these sessions, their similarity to ADL leader- 
ship seminars is unknown. Due to a large amount of personnel turnover in 
the district, principal seminars/meetings were not held during the 1983-84 
school year* 

Principal/ teacher conferences . These formal conferences provide an 
opportunity, following each round of observations, for the principal and 
teacher to discuss data on all of the critical student behaviors, identify 
any opportunities for improvement, and agree to improvement plans. BSC 
suggests that principals conference with each teacher at least three times 
a year. A conference form was developed for teachers and principals to 
record conference data and strategies for improvement. 

During the field test year, BSC assistance related to principal /teacher 
conferences was provided through a principal seminar and also through 
individual meetings with principals and assistant principals. The Basics 
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Plus principal followed the first round of observations with conferences 
(using a portion of the ADL conference form). The second round was followed 
by pairs of teachers discussing strategies (no conference form was used) , 
and the final round of observations was followed by the district's mandated 
evaluation conference (using the ADL engagement rate form and the district's 
evaluation form). The assistant principal at the other elementary school 
also conducted conferences following observations but used the engagement 
rate form rather than the conference form. Student engaged time was the 
only classroom variable discussed. Overall, BSC technical assistance in 
this area was not successful. This was due, in part, to the superinten- 
dent's reluctance to support the use of the conference form. He was 
concerned about possible confusion between the ADL conference form and the 
district's evaluation form. At one point, the superintendent considered 
developing separate conference schedules (and procedures) , but never 
followed through with the idea. 

The following year, 1982-83, the Basics Plus principal used the 
engagement rate form once alone, and then again with the district's 
evaluation form during teachers' annual evaluations. Principal/teacher 
conferences were not conducted at the other elementary school that year, or 
at either elementary school the next year, 1983-84. 

Super in t endent / princ ipal conf e rences . The ADL leadership plan 
specifies that the superintendent hold several formal supervisory confer- 
ences with principals to review school and classroom data and to discuss 
improvement opportunities. 

During the field test year, the BSC linker met individually with 
central office staff and the principals to discuss the conferences. The 
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director of special program reported conducting two conferences with the 
Basics Plus elementary school principal and the assistant principal of the 
other school. The director of special programs held the second round of 

i 

conferences reluctantly because she felt the first round had been of little 
value. However, the Basics Plus principal indicated .that the conferences 
were helpful. No additional superintendent/principal conferences were 
conducted during the 1982-83 or 1983-84 school years. 

It is not surprising rhat BSC's technical assistance did not facili- _ 
tate the implementation of superintendent /principal conferences. District 
support was limited . and principals did not use the structured principal/ 
teacffer conference forms to identify improvements needed in classroom 
variables and instructional leadership functions. 

^Differentiated inservice . Seminars . principal/ teacher conferences, 
superintendent/principal conferences, and informal observations are means 
for district leadership to collect and synthesize information about the 
performance of principals and teachers, and to identify individual needs 
related to Instructional improvement. The leadership plan suggests that 
the district office be proactive in providing for small or large group 
inservice based on diagnosed needs • No such Inservice was provided to 
principals or teachers from 19&1 to 1984 ♦ 

Technical Assistance for Evaluation 

From the outset. BSC staff intended to provide technical assistance to 
assure that districts focused on the principal benefits of the Innovation — 
improved instruction and Increased student achievement. Therefore, technical 
assistance emgjiasized the long-term impact of implementation events and 
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procedures. While limitations on BSC resources hsve prevented the systematic 
collection and analysis of data on impact, data were gathered from partici- 
pants about their perceptions of changes in roles — especially changes which 
reflected the leadership plan. Both BSC and districts also gathered data 
on student achievement. In general, BSC shared the date with the field 
test districts for planning purposes. However, formative evaluation of ADL 
was of little concern to the Delaware district since implementation dropped 
off after the field test. 

Role changes . One AOL objective is to have district staff carry out 
activities defined by the leadership plan. If staff change their behavior 
to conform more closely to the functions specified in the plan, then the 
program has had an impact. The following discussion of role changes Is 
based primarily on interviews which BSC staff conducted in the spring of 
1982, 1983, and 1984. (A complete list of interviews is found in Appendix 
A.) 

Since the superintendent brought ADL to the district, principals felt 
that he had always been an instructional leader. Therefore, when asked if 
the superintendent's role had changed, they indicated it had changed very 
little or they were undecided. The deputy superintendent felt that her 
role changed somewhat; the director of special programs felt that her role 
changed a .great deal but that this change was not maintained. Interestingly, 
teachers felt that the district's role changed somewhat or a great deal, 
and noted inservice devoted to instructional planning as the major district 
change. 

The principal of the Basics Plus elementary school felt that her role 
changed a great deal, particularly with regard to observations, the 
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conference model, and attending to classroom variables. This principal 
co-authored an article about the development effort, "Improving Instruction 
Through Research-Based Staff Development" (Educational Technology. 1980) . 
in the article she testified to her own role change and that of her staff: 

• 

The program affected the teacher's role with students, other 
teachers, and the principal. In addition, the project enabled 
the principal to look for many more specific student and 
teacher behaviors in the classrooms, such as the teacher 
establishing anticipatory set, objective, and purpose, and 
the students' performing "engaged" behaviors. It facili- 
tated the use of descriptive vocabulary concerning these 
behaviors. The principal referred to specific data, rather 
than opinions, (p. 42) 

The director of special programs reported that this principal' s *ole 
changed somewhat. Two of the teachers interviewed indicated that the 
principal's role had changed a great deal or somewhat. They also felt that 
' their own roles had changed a great deal. Both mentioned the management of 
time as the greatest area of change. The third teacher was undecided about 
the principal's and her own role change. She began her teaching career 
during the year the district began cooperating with RBS in developing the 
program. Therefore, she felt she had no standard of comparison; 

The principal of the other elementary school felt that her role 
changed somewhat even though she never conducted ADL classroom observations 
or "conferences (the assistant principal accomplished these tasks during the 
field test). The director of special programs agreed with the principal's 

# 

assessment. However, the three teachers Interviewed at that school were 

< 

undecided about the principal's role change, or felt that it changed very 
little or somewhat. Calling attention to the "ADL variable*" was the only 
change mentioned. These teachers assessed their own role change as somewhat 
(two teachers) or very little (one. teacher) . The areas of change mentioned 
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were classroom management and attention to prior learning strengths and 
weaknesses. The principal reported very little role change for teachers in 
her building. District staff Indicated that teachers in both schools only 
changed somewhat. <•» 

Student achievement . ADL's primary objective is improvement of 
student achievement. The Delaware school district began to cooperate with 
RBS in program development for the express purpose of raising low achieve- 
ment. BSC, for its part, did not anticipate dramatic gains in achievement 
in the first few years due to the incompleteness of many program elements. 
Nevertheless, BSC collected student achievement data for the district as a 
whole, by grade, each year. ^ 4 

ADL was first implemented in the school district in the 1978-79 school 
year. In the fall of 1978, student achievement at most grade levels was 

» 

slightly higher than the national norm but lower than the state norm. 
However, in the spring of 1979, after one year of implementation, all 
grades registered impressive Increases. (See Appendix C for a discussion 
of the significance of achievement gains.) The average gain on the total 
battery of tests was +13 NCEs. Gains ranged from +0 to +22 NCEs. These 
gains are an encouraging sign of ADV.s potential for raising student 
achievement, but they should be viewed cautiously. It Is generally unwise 
to compare test results from tests administered at different points in the 
same school year — in this case fall 1978 ro spring 1979. The comparison is 
discussed here because the state department did not mandate testing at all 
grades in the spring of 19J&. Overall, student achievement dropped slightly 
from the spring of 1979 to the spring of 1980. ^The average change in 
achievement was -3 NCEs. Changes ranged from -10 to +3 NCEs. 
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Te«*t results for grades erne through six across five years (1980-84) 
are suaaarlsed in Table 3 in terms of nonpi curve equivalents (NCEs). The 
table also shears the change in HCEs registered froa one year to the next. 
The scores in the horizontal rotrs represent different groups of students in 
each successive year. 

Table 3 

Student Achievement Results: 1980-1984 
Delaware School District 



Grade 


Scores* 




Gains/Losses 




1980 


1981 


.1982 


1983 


1984 


80-81 


81-82 


82-83 


83-84 


1 


61 


65 


65 


64 




+4 


0 


+1 




2 


67 


65 


65 


67 


66 


-2 


0 


+2 


-1 


3 


58 


63 


63 


64 


62 


+5 


0 


+1 


-2 


4 


63 


64 


65 


65 


66 


+1 


♦1 


0 


+1 


5 


60 


62 


69 


69 


62 


+2 


+7 


0 


-7 


6 






60 


-64 


62 






♦4 


-2 


Mean 


62 


64 


65 


66 


64 


+2 


< +1 


< 

+1 


-2 


















j 


i , ._ . . 



a • 
Scores represent results of California Achievement Tests (total battery; 

In Normal Curve Equivalents (NCEs) for 1980-1983, and Comprehensive 

- Test of Basic Skills (total battery) fo$ 1984. Means are based on 

scores for grades 1-5 for 1980-83, and grades 2-6 for 1984. 



From 1980 to 1983 (see Table 3), achievement rose slowlybut steadily. 
In the spring of 1984, however, achievement seemed to drop slightly. This 
drop coincided with the departure of the superintendent and the principal 
of the Basics Plus elementary school. Both were strong supporters of the 
program. It should be noted that, although all test scores are reported as 
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NCEs, the state substituted the Comprehensive Test of Basic Skills (CTBS) 
for the California Achievement Test (CAT) in 1984. 

Technical Assistance and Institutionalization 

BSC linkers pursued several strategies to sustain implementation and 
foster institutionalization. The first major strategy BSC used to foster 
institutionalization in the Delaware district was to relate ADL training 
during principal seminars to other district activities and concerns. For 
example, during one principal seminar the BSC linker related ADL to mastery 
learning. At the time, implementation of mastery learning was one of the 
superintendent's highest priorities for the district. However, this 
strategy was not successful as the superintendent did not continue to 
include issues directly related to ADL in principal meetings as the level 
of implementation decreased. 

BSC's second strategy was to encourage the incorporation of engagement 
race observations into principals' annual teacher evaluations. Although 
one principal adopted this practice to some extent (i.e., engagement rates 
were discussed in the evaluation conference but were not entered on evalua- 
tion forms), the idea was opposed by other principals, assistant principals, 
and a select group of teachers. This strategy was also not successful as 
the superintendent relented to the pressure. The principal who adopted the 
practice continued to include engagement rate data in annual evaluations 
until her departure in 1983. 

In conclusion, ADL practices were not mandated in the Delaware district, 
and the one administrator who provided assistance with implementation, the 
director of special programs, did not have line authority over principals. 
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Thus, the key factors needed for institutionalization, according to Miles 
(1983), administrative support, demonstrated commitment, and pressure, were 
absent. In addition, the lack of stability omons program leadership and 
staff negatively affected any opportunity for institutionalization. 

Nevertheless, there are encouraging signs of institutionalization. 
Teachers In the Basics Plus school have continued to scan their own classes 
for engagement rates— without the assistance of the new principal. In 
addition, the district has been successful in integrating into district 
routines the practice of examining prior learning data in planning instruc- 
tion. Before the start of each of the three school years, the district 
requested reorganized classroom data from the state department. DPI worked 
with administrators in the district on attending to prior learning. A 
cross-fertilization of ideas between the district and the DPI seemed to 
occur: DPI later delivered throughout the state training related to prior 
Learning. 
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CHAPTER SIX 
CONCLUSIONS AND OBSERVATIONS 

The previous chapters of this report described BSC experiences with 
strategies designed to foster implementation and institutionalization of 
ADL in three districts. The first section of this chapter presents some 
conclusions concerning the five major research-based strategies which BSC 
pursued. The second section contains some general observations on the 
barriers to and facilitators of iapleaentation and institutionalization. 
As noted in chapter 1, these conclusions and observations are not presumed 
to-be definitive. Rather they are important reflections on the component's 
experiences which BSC believes are worthy of the consideration of others 
who are working to enhance the effectiveness of schools and classrooms. 

Conclusions Concerning Major Strategies 

the five major strategies BSC pursued in its capacity building efforts 
were described in chapter 2: 

• limit the number of highly specified implementation processes 
and materials which are essential to a faithful implementa- 
tion of ADL 

• orient, plan, and train following a top-down sequence 

• use innovation-specific implementation events to help 
districts develop the general planning and organizational 
skills needed for implementation of the innovation 

• provide on-site technical assistance following training 

• consider the probable long-term impact of early design and 
implementation activities. 

BSC's conclusions about each of these strategies are discussed below. 
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Llait Specificity 



The intent of this strategy was to provide opportunities for iaplesten- 



tors to create their own distinctive means of carrying out many of the ADL 
activities and thus develop ownership in the improvement effort. Rela- 
tively early in its development work, BSC began to question the value of 
this strategy for the following reasons: (1) in many cases districts 
had little time for these supporting activities; (2) practitioners were 
more Interested in rights of review and approval than in contributing to 
the design of experimental methods and materials; and Of when practi- 
tioners did participate, this participation did not always lead to a sense 
of ownership (Graeber & Helms, 1983)* 

In planning for the field test, the superintendents agreed with BSC's 
assessment that in the absence of BSC specified processes or materials, 
needed training or implementation events were not likely to occur* Thus, 
as noted in chapter 3, BSC devoted much time in the summer of 1981 to the 
design and development of ADL elements that had previously been reserved 
for local users. 

In order to facilitate district use of the variables management 
process, three implementation activities were specified* BSC provided a 
rationale, reasonable methods, and training for: principal seminars, 
principal/teacher conferences, and superintendent /principal conferences. 



As indicated in chapter 5, experience in the three districts proved that 
BSC's development efforts were beneficial and that the activities supported 
the districts' use of the improvement cycle* For example, principals 
viewed their conferences with the superintendent and the seminars as 
supporting and benefiting instructional improvement. Similarly, most 
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teachers reported that their structured conferences with principals 
contributed to instructional improvement. And, linkers,/ central office 
staff, and principals reported that the principal/teacher conferences 
fostered teacher use of the variables management process in planning and 
executing classroom Instruction and management. 

To facilitate the use of the variables management! process, BSC 
developed training material for another implementation activity, the 
provision of differentiated inservice at the building and district levels. 
Although differentiated inservice was, not a pre-desigaed program element, 
BSC expected that Issues raised in seminars and conferences would lead to 
the development of differentiated Inservice if the district was attending 
to its own renewal and reform. The concept of differentiated inservice was 
new to all three districts, and relatively little differentiated inservice 
was provided. The pervasive notions of equity and choice with respect to 
inservice (i.e., all principals or teachers are treated alike, or any 
differentiation is the result of a choice made by the intended beneficiary) 
seemed very difficult to change. The expected logistical difficulties and 
contractual arrangements that specified who determined the content of 
inservice were formidable barriers to differentiating inservice according 
to individual need. Differentiated inservice is an intended outcome of ADL 
implementation that was difficult to achieve. 

In retrospect, BSC's experience suggests that development of methods 
and materials which specified a way, albeit not the only way, of carrying 
out leadership activities directed toward use and support of the variables 
management system was beneficial. But what of the ownership local develop- 
ment was to build? The component's experience suggests that, at least for 
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an Innovation aa comprehensive as ADL, there are many opportunities for 
adaptations despite relatively complete predesign of materials. BSC 
observed that the sense of ownership (as well as understanding) was built 
as people invested tine in the implementation and when the implementation 
was successful. BSC experiences in the three districts also showed that 
when the superintendent clearly required that certain activities be under- 
taken and provided needed support and monitoring, central office staff, 
principals, and teachers utilized predesigned materials and procedures to 
achieve changes in classroom processes and in student achievement. 

Install Top-Down 

BSC adopted a top-down installation model (superintendent, principals, 
teachers) for two main reasons. First, the component sought to garner the 
support of the district leadership by recognizing their leadership position. 
Second, BSC saw this top-down strategy as an economical means for dissemi- 
nating ADL within a district, through top-down turnkey training. 

BSC is still convinced, perhaps more convinced than ever, of the value 

of top-down installation in gaining the support of district leaders. As 

Huberman (1983) notes? 

....administrators, both at the central office and building 
levels, have to go to center stage and stay there if 
school improvement efforts are to succeed. More nondlrec- 
tive strategies can work ...but are poorer bets; they 
amount essentially to playing dice with the fate of an 
innovation, (p. 27) 

A top-down approach places administrators at center stage. 

Given the structure of school districts (e.g., frequently principals 

are responsible only to the superintendent; lack of coordination between 

staff with curriculum, testing, and inservice responsibilities) it seems 
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unlikely that district staff other than the superintendent could command 
the resources, support, or authority required to implement ADL or any 
comprehensive innovation. Without the superintendent's active participa- 
tion, implementation is likely to fail. 

The second proposed benefit of the top-down model was to develop the 
district's own capacity to train principals and teachers. BSC still 
believes that this notion is sound. In addition to the economic advantages, 
the strategy also helped establish local personnel as instructional 
leaders. There were practical problems with the turnkey training strategy, 
but BSC maintains that the strategy's benefits warrant the effort to solve 
the problems. 

In many cases, central office staff and principals seemed reluctant to 
turnkey their training because of the high level of knowledge ,and skill 
that appeared to be required. In response, BSC provided them with training 
scripts and videotapes of the technical aspects of training. It also 
seemed helpful to model the training for the central office staff and 
principals and to give them time to plan and practice the training in small 
groups. . Districts used other strategies as well, such as having two 
principals, or a principal and a member of the central office staff, share 
their training responsibilities. 

A second difficulty with the turnkey train ng strategy occurred in 
districts with few central office staff. No time was available for these 
staff to be trained and then to turnkey their training to principals, given 
the numerous responsibilities of the central office staff in a small 
district. In these instances, central office staff and principals were 
trained in the same sessions. It was possible to schedule a few critical 
planning/decision-making sessions with central office staff only. 
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Finally, superintendents were reluctant to spend the time required for 
their own training, although "they readily agreed that aany of the required 
decisions deaanded their understanding and attention. BSC suggests that 
training can be tailored sore specifically to superintendents and that 
attendance at an academy exclusively for superintendents might be a practi- 
cal and logical prerequisite to district adoption of the program. 

BSC's experience confirmed the value of the top-down installation 
process, in spite of the model's difficulties such as its dependence on 
strong administrative leadership, end its vulnerability to central office 
politics and existing policies (e.g., policies concerning principal 
autonomy). However, it must be recognized that ADL is designed to facili- 
tate communication from the bottom up so that each level of the system can 
provide informed support for the levels under its supervision. Hp-down 
installation without provision for feedback and support from other levels 
would contradict the central philosophy of ADL and would surely fail in 
practice. ADL provides for this feedback and support through structured 
conferences and seminars. 

Begin Ins tallation at the District's Current Level of Organiza tional 
Development ~ ~ 

BSC decided that in districts with weak organizational skills (e.g., 
planning), as is frequently the case in low achieving districts, BSC would 
strive for improvement of these generic skills as the district worked to 
install and implement ADL. To not work with such districts was inconsis- 
tent with the component's mission. The other alternative, of first working 
with a district on improving its organizational skills, would have unduly 
lengthened the instructional improvement process and might have distracted 
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staff from the procedural or technical changes needed for the improvement 
of Instruction (e.g., alignment of curriculum and testing). 

Although the three field test districts appeared capable of carrying 
out many logistical activities (e.g., bussing pupils, providing supplies) 
with some facility, their planning and delivery systems for inservlce and 
instruction presented opportunities for improvement . At the central office 
level, the most frequent opportunities for improvement were in planning 
projects and coordinating them across departments or divisions. The 
planning capability of district personnel was enhanced, but it took time to 
establish the discipline of planning and following through. Many installa- 
tion and imp lamentation events in the districts were delayed because 
planning was not completed on time. 

Many principals needed to improve their planning and delivery of 
teacher training and their conduct of conferences. When it was necessary 
for linkers to assist with process skills, installation and implementation 
proceeded more slowly. For example, linkers found it necessary to provide 
special training to principals on conferencing, including time for practice 
with feedback. Although % this training delayed -implementation, the extra 
assistance proved to be valuable in the development of needed skills. 

Despite their need to develop organizational skills, the three dis- 
tricts succeeded in installing and maintaining ADL, and student achievement 
improved. As Pincus and Williams (1979) suggest, external linkers need to 
be aware of a district's general level of organizational development. 
BSC experience suggests that organizational skills can be fostered con- 
current with specific instructional improvements. However, BSC agrees with 
Rosenblum and Louiz (1981) and Pincus and Williams (1979) that some 
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stability In district leadership and a minimum of organisational skills are 
required before a district can effectively innovate—especially if the 
Innovation is coaplex. BSC's experience confirmed the comment that while 
"skeptics claim that only schools that are ready" to change can succeed from 
the strategies. . .schools that are not advantaged can benefit in aajor ways 
froa very aodest amounts of assistance." ("Two large-scale," 1982, p. 2). 

Provide Technical Assistance 

BSC's reading of the literature suggested that district success with 
implementation would require on-site technical assistance after the 
completion of initial training in the classroom variables. BSC conceived 
that this technical assistance to the district should Include one or more 
of the five components of effective teacher inservice suggested by Joyce 
and Showers (1980), as needed: presentation, modeling, practice, feedback, 
and coaching. 

In reviewing its initial training of central office staff and princi- 
pals, BSC concluded, as did Joyce and Showers (1980), that "it appears 
wisest to include several and perhaps all of the training components" 
(p. 384). There were Instances when one or more of these components were 
omitted from initial training (e.g., not providing principals with time to 
practice their training of teachers) because of a lack of time or because 
BSC presumed a level of skill that in fact did not exist. In these cases, 
the omission was reflected in the trainees' less than satisfactory applica- 
tion of that training. 
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As reported in chapter 5, BSC's technical assistance during implemen- 
tation was helpful to the districts and compensated for some of the short- 
comings of the Initial training. It is interesting to note that the 
technical assistance provided to districts during implementation did Indeed 
fall into categories that are essentially the same as the Joyce and Showers 
components of effective training. On site, the linker: 

• presented new perspectives on the initial training, elabo- 
rating on the rationale or recalling parts of the training 
and relating it to the district's current applications 

e demonstrated an activity, if needed 

e provided more practice 

• gave feedback 

• observed the district's transfer and provided further 
on-site assistance as needed. 

Usually only one or two of these steps were used to fine tune a given 
process. However, the linkers used. each of the steps at some piint during 
the three years. 

BSC perceives that the amount of technical assistance provided in two 

districts (approximately one day per week during the field test year and 

one day every other week during the following two years) was higher than is 

"typical" during implementation of an innovation. The question of how much 

technical assistance to provide was always a BSC concern. How do linkers 

provide needed assistance without fostering dependency? Or, when do 

linkers' attempts to assist proactively work against building local 

capacity? Some of the guidelines BSC linkers developed are: 

• Require that district personnel be present to contribute to 
planning sessions. Do not plan, In their absence. Whenever 
possible, show how planning can be guided by training 
materials available to the district. 
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4 

• Model a Cask when the district has specific plans to carry 
, out the task in the near future. If the district will not 

be carrying out the task on its own, insist that the 
learners take at least soae small role in the task. v * 

I , P 1 

• Have the district present installation and implementation 
plans and timelines to principals and teachers , thus con- 
veying district, rather than external agency , ownership* 

e Continually assess the need for external assistance . If 
there is a true lack of knot-how, provide assistance. If 
external participation is valuable because it creates a 
priority for the Innovation in a setting with stany demands, 
provide the technical assistance but also strive to increase 
the district f s commitment to the implementation. If district 
personnel are unwilling to implement, perhaps because 
11 administrators are unwilling to take on the conflicts 
involved" (Huberman, 1983, p. 26), consider discontinuing 
technical assistance unless there Is a change in the 
behavior of the district leadership. 

Consider Long-Term Impact 

BSC 1 s fifth major strategy was the attempt. to incorporate lessons on 
educational change into the design, Installation, and implementation of the 
innovation. The component was particularly Interested in the probable 
long-term impact of the program' s design and implementation tasks on 
implementation and Institutionalization. The lessons followed during 
program design were reported in Graeber and Helms (1983). Lessons that 
were followed with respect to installation and implementation are summar- 
ized below. When the lessons were followed, they fostered institutional iza 
tion; however, barriers Impeded implementation of some of the lessons 

BSC followed the educational change literature 9 s suggestion that 
districts use local funds or resources to support the innovation. Although 
BSC's training and technical assistance were supplied without charge, 
districts had to fund inservice time for principals and teachers, the 
duplication of materials (after the field test year), and other activities 
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related to content management (e.g., reorganizing test data). Districts 
accepted this strategy, and BSC found that a district's willingness to use 
its resources was a Measure of commitment. 

i 

•BSC linkers attempted to have districts avoid duplication of effort 

• r 

between the innovation and existing processes. Duplication of effort was 
always- an issue for the teacher observation and conferencing process. Each 
of the three states had its own forms and/or requirements for classroom 
observations and teacher- evaluations. Although the specificity of the 
procedures and forms varied, none Involved systematic data collection on 
the critical classroom variables that is part of ADL's observation/ 

/ 

conferencing process. The three districts opted to keep the teacher 
evaluation system separate from tti^£provement effort for the first 
several years. This raised some concerns.' When the systems were kept 
separate, principals objected to conducting two sets of observations but 
•felt 'obligated to do so. Given the infrequency of state mandated observa- 
tions (one or two a year), the total number of observations required per 
year was only a legitimate concern irf schools with large numbers of 
teachers. Having dual observation systems raised two other issues. First, 
if the systems were kept separate, should principals ignore the improvement 
observations when completing the state mandated evaluations? Second, if 
the improvement effort observations were directed at the critical vari- 
ables, shouldn't the formal evaluations address the same factors? Although 
all three districts suggested that the observation systems be merged after 
a yaar or two, this was not accomplished in any of the three districts. 
The issue was complicated by the districts' contractual arrangements with 
teachers, as well as by state evaluation requirements. 
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Somewhat more progress was achieved with another lesson on educational 
change, which was to encourage districts to incorporate ADL activities into 
district policies and procedures* The willingness of the district leader- 
ship to actively champion the changes in policies and procedures was 
critical. At times, this involved negotiating changes in staff contracts 
(e.g., to legitimise use of teacher "prep periods 9 ' for principal/teacher 
conferences). However, even when districts made alterations in their 
policies and procedures, the continued active support of the superintendent 
was essential for institutionalization to occur. 

Observations 

BSC's experience has identified a number of conditions in districts 
which facilitate or hinder implementation and institutionalization. This 
section includes some observations on the most salient of these conditions: 
commitment, coupling, districtwide implementation, and administrative 

leadership. 

Commitment 

There is agreement in the educational change literature that teacher 
and administrator commitment to an innovation is positively ..corral n ted ta 
outcomes of the implementation. Questions of how and when the commitment 
is developed remain unanswered. 

BSC originally adopted the strategy of limiting prescriptiveness of 
the innovation and encouraging local development to build user commitment. 
That strategy seemed unreliable. Further, BSC, like Huberman (1983), 
frequently found that change is not always in the self interest of district 
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personnel. Often during change, "one person 1 s 1 strategy 1 for school 
improvement collides with another person's 1 strategy 1 for avoiding loss of 
status or freedom or benefits" (Huberman, 1983, p. 23). Huberman notes 
that, as a result, the change process is taken up with political bargaining 
that draws energy away from implementation. This occurred in the Pennsyl- 
vania and New Jersey districts. In the Pennsylvania district, the struggle 
continued throughout the three years, and the innovation suffered. In New 
Jersey, the superintendent was adamant in calling for implementation, which 
initially may have caused low levels of principal and teacher commitment * 
However, implementation took place, and in comparison to the Pennsylvania 
district the prospects for Institutionalization of ADL seem brighter. 

BSC experience suggests that attempting to win commitment for an 
innovation prior to implementation may be unrealistic, and that commitment 
from central office staff, principals, and teachers develops with mastery 
of the innovation and with success with its use. A coiranitment from the 
superintendent to sincerely back the innovation may be a more reasonablr 
expectation for external linkers. 

Coupling 

A much debated issue in recent literature is whether or "not the loose 
coupling so frequently observed in school districts is beneficial to 
implementation. BSC experience confirms the Wilson and Corbett (1983) 
observation that "if certain practices are effective and deemed worthy of 
widespread use, then tighter linkages are apparently the structural condi- 
tions that can best promote their implementation" (p. 102) ♦ 

This view seems consistent with Huberraan's (1983) observations that, 
for an innovation to succeed, administrators need to exert "strong and 
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continuous" pressure on teachers to udopt the new practices. It is BSC's 
observation that this pressure must be exerted not .inly on teachers but 
also on principals and central office staff.. 

BSC^wcperience further suggests that districts who vish to tighten 
couplings can do so, although much time and effort are needed to counteract 
staff resistance. Loose coupling does not have to be accepted as a given 
condition. 

Districtwide Implementation 

Another factor related to commitment to and stkcess of an innovation 
is the extent of the initial implementation within \ district. Many of the 
superintendent's decisions and the central of flc/stalf *s tasks are essen- 
tially the same regardless of the number of schools involved in the imple- 
mentation. It is difficult for BSC to determine the relationship between 
districtwide implementation and commitment. Does greater initial commit- 
ment lead to districtwide implementation? Or does the decision to imple- 
ment districtwide result in greater commitment from the leadership? In 
either case, BSC observed greater administrative commitment and leader- 
ship when implementation was, districtwide. This observation is consistent 
with Miles' (1983) observation that as the percent of users approaches 100 
percent of the potential users, the likelihood of institutionalization 
increases. 

"Pilot site" efforts in a district were difficult to sustain and 
spread because (1) they required relatively large investments of central 
office time for the benefit of a few; (2) they frequently raised divisive 
speculation about why some buildings volunteered or were selected and 
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others were not; and (3) implementation problems sometimes became rallying 
points from which to attack the feasibility of the innovation rather than 
opportunities to improve the pilot program or alter the context to achieve 
an effective implementation. 

Leadership and Institutionalization 

It seems clear to BSC that the cluster of factors discussed by Miles 
(1983), administrative commitment, pressure* and support, is essential to 
institutionalization. In the Delaware district, where key administrators 

i 

left the district the year after the field test, very little of the innova- 
tion was sustained. In the Pennsylvania district, where administrative 
pressure, support, and assistance, if not commitment, were weak, progress 
toward institutionalization was slow and tentative. And, although pros- 
pects in the New Jersey district are brighter, -the superintendent himself 
suggested that the decline in level of implementation in the 1983-84 school 
year reflected his lowered support and pressure. This suggest, that 
"institutionalization" is highly dependent on the continued support of 
district leaders. BSC questions whether institutionalization is ever 
routine In the sense that little energy is required to sustain the Implemen- 
tation. 

District support for an innovation is probably never wore tenuous than 
during turnover In leadership. While It is true that a district's chances 
of maintaining an inflation after loss of a key leader may be enhanced if 
the innovation has been built into district policy and procedures, it is 
also true that such policy and practice can and may well be changed by new 
leadership bent on making its own mark. Given the tendency districts have 
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to meander from innovation to innovation (Runkel, Wyant, Bell, & Runkel, 
1980) and the tendency of new brooms to sweep clean, the chances of sus- 
taining an implementation in spite of changes in key leadership seem slim. 

! I ! i 
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Appendix A 
Data Sources on Program Implementation 



Summary of Data Sources on Program Implementation, 1981-84 
Pennsylvania School District 



th — - - 

District 


Level 


Data 
Source 


Date 
Administered 


Interviewees 


Period 
Covered 


Pennsylvania 


District 


Installation Interview 

Interview 
Interview 
Interview 


Winter, 1981-82 

Spring, 1982 
Suisse r, 1982 
Spring, 1984 


Superintendent and 

6 District Staff 
7 District Staff 
Superintendent 
3 District Staff 


1981-82 

1981-82 

1981- 82 

1982- 83 & 
1983-84 




School 


Installation Interview 
Participatory Supervision 

Interview 
Interview 
Questionnaire 
Interview 
Interview 
Interview 


Winter, 1981-82 
Winter, 1981-82 

Spring, 1982 
Spring, 1982 
Sumner, 1983 
Spring, 1984 
Spring, 1984 


5 Principals 
S Principals 

5 Principals 
5 Principals 
5 Principals 
12 Principals 
5 Principals 


1981-82 
1981-82 

1981-82 

1981- 82 

1982- 83 

1983- 84 
1982-83 & 

1983-84 


— . — .■ * 


Classroom 

■ 


Interview 

Questionnaire 

Interview 


Spring, 1982 
Spring, 1982 
Spring, 1984 


12 Teachers 
46 Teachers 
34 Teachers 


1981-82 

1981- 82 

1982- 83 & 
1983-8* 
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Summary of Data Sources on Program Implementation, 1981-84 

New Jersey School District 



1 


I 


Data 


Date 


1 


Period 


District 


Level 


Source 


Administered 


Interviewees 


Covered 


New Jersev 


Distrir £ 




CttJllf X70X 


Ciirmf^ «^ f" ortrf £»n f' art /I 
DUpcl micuUtrllL OUU 


1 481 ~89 










XVI UXoLiXvL DLHXl 








Interview 


Spring, 1982 


Superintendent 


1981-82 






iutervxew 


Cnr^no 1QOO 

spring f i7oz 


iu uis crxec %> car r 


1 QUI —AO 
1701-0/ 






Interview 


Winter, 1983-84 


Superintendent 


1982-83 






luLcl View 




CnnAi*4 nf anil anf' 






School 


Installation Interview 


Fall, 1981 


9 Principals 


1981-82 








Winter 1981— 82 










Interview 












Interview 


Spring, 1982 


9 Principals 


1981-82 






Questionnaire 


Spring, 1982 


9 Principals 


1981-82 






Interview 


Fall, 1983 


8 Principals 


1982-83 






Interview 


Spring, 1984 


8 Principals 


1983-84 




Classroom 


Interview 


Spring, 1982 


20 Teachers 


> 1981-82 






Questionnaire 


Spring, 1982 


116 Teachers 


1981-82 






Interview 


Winter, 1983-84 


18 Teachers 


1982-83 A 












1983-84 



1G2 
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Summary of Data Sources .on Program Implementation, 1981-84 

Delaware School District 



District 


Level 


Data 
Source 


Date 
Administered 


Interviewees 


Period 1 
Covered j 


Delaware 


District 


Installation Interview 

Interview 
Interview 
Interview 
Interview 

Interview 


Summer, 1981 

Winter, 1981-82 
Spring, 1982 
Sumer, 1982 
Spring, 1984 

Spring, 1984 


Superintendent and 

3 District Staff 
1 District Staff 
1 District Staff 
Superintendent 
Superintendent 

1 District Staff 
(former) 


1981-82 

1981-82 
1981-82 

1981- 82 

1982- 83 & 
1983-84 

1982-83 & 
1983-84 


j 


School 


Installation Interview 
Participatory Supervision 

Interview 
Interview 

Questionnaire 
Interview 


Summer, 1981 
Fall, 1981 & 
Winter, 1981-82 
Spring, 1982 

Spring, 1982 
Spring, 1984 


2 Principals 
2 Principals 

2 Principals 

1 Asst. Principal 

1 Principal 

2 Principals 
(1 former) 


1981-82 
1981-82 

1981-82 . 

1981- 82 

1982- 83 & 
1983-84 


- t 


Classroom 


Interview 
Interview 
Questionnaire 
Interview 


Fall, 1981 
Spring, 1982 
Spring, 1982 
Spring, 1984 


4 Teachers 

5 Teachers 
28 Teachers 

6 Teachers 


1981-82 
1981-82 

1981- 82 

1982- 83 & 
1983-84 
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Appendix B 
Summaries of Program Implementation 
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SUMMARIES OF PROGRAM IMPLEMENTATION 



The following three tables summarize data on implementation of ADL in 
three consecutive school years (1981-84) in three school districts. The 
tables are organized by four of the five critical program elements discussed 
in chapter 5 (i.e., critical classroom variables; principal seminars; princi- 
pal/teacher conferences; and district/principal conferences) . Differentiated 
inservice is not included due to the lack of comparable data. 

Each table presents data from four sources: Lhe BSC, the district 
(e.g., superintendents, resource teachers, curriculum specialists), school 
(principals), and classroom (teachers) levels. Attempts were made to ques- 
tion educators at each level about the same topics, i.e., triangulate, in 
order to establish the -reliability of the data. In general, the tables 
reveal a good deal of agreement between levels. Data sources Include: BSC 
observations and contact reports; superintendent interviews; principal and 
teacher interviews and questionnaires; and principal/ teacher conference 
forms. 
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Table B-l 

try of Progras Implementation, 1*81-84 
Pennsylvania School District 



Program El eaten t 

epilation of Class- 


1981-82 

0 


Tear and Level of Response 

1982-63 


1983-*4 C 


rat* 


DISC. 


Seal. 
(*-5) 


Clsra. 
(Jf-46) 


BSC 


Dlst,. 
(Ifl) b 


Schl. 


Clsra, 
CW-10) 


BSC 


Mst. 


Seal, 


Clsra. 


room Data 

f 

m Variables attended , 
to (7 t C, PL, AP) d 


























all* 


All 


All 


All 


T(l Sch.) 
C(2 Schs.) 




T(l Scb.) T(l Sch). 
C(2 Schs.) C(2 Schs.) 


All 


All 


All 


All 


# Por which subject 
areas 


M 
R/LA 


M 

R/LA 


N 

R/LA 


M 

R/LA 


N 
R/LA 




M 
R/LA 


R/LA 


M 

R/LA 


M 

R/LA 


H 
R/LA 


M 

R/LA 


# Knottier of class*- 
rooms involved 


(5 Scb.) (5 Sfcb.) 
All All 


(5 Scb.) 
All 


(5 Sch.) 
All 


<2 Sch.) 
All 




(2 Scb.) 
All 


(2 Scb.) 
All 


All 


All 


All 


All 


o Participating 
Supervisors 


Prin. 


Prin. 


Prin. 


Frin. 


Prin. 




Prin. 


Prin. 


Prin. 


Prin. 


Prin. 


Prin. 


e Mean number of 
observations 


3 


3 


3 


7* 


2 

(1 Sch.) 




2 

(1 Sch.) 


4 

<1 Scb.) 


4 


4 


4 


5* 


e Range of observa- 
tions 


3 


3 


3 


2-15* 


2 




2-8 


2-8 


4 


3-7 


3-7 


2-7 



^Tbe superintendent, resource teachers, and curricula* specialists. 
c Curricula* specialist. 

%lt!l£?. eiP * 1 **' teaCh ' V lBterVieV8 "" fr0 " 17 "boo!.. District data fr« Director of Instruction and t~ curricula-, 

*I * Tlae '' C - Content; PL - Prior Learning; AP - Academic Performance. 

g C , r2t":2.n 1 LSrI^.?° Bt#,,t (Pri " rlly COV '" 8 ' ln Mth> «- r~r; -ini^l attention to acad«ic perfo«ance. 

Interviewees may have confused ADL observations with non-ADL observations. 

*° tei S2c^; a £^^.S*^ W - 8 ° d -Perintendent interviews; principal and t.acSer i n c ervlws and eu.stionnaire. ; 



Table S-l (coot*) 
Pennsylvania School District 




Program Eleaent 



Principal Leadership 
Seminars 

e Seminar leader(s) 



e Seminar frequency 



Frinc ipal /Teacher 
Conferences 

e Kumber of parti- 
cipating teachers 



• Participating 
Supervisors 



• Mean number of 
confs. per teacher 



• Range in number 
of conferences 



# Variables discutsed 



*SC . Dist. Schli Clam. 

(*-7) Qf-5) (*-t6}_ 



BSC 



BSC 



BSC 



HA 



11% 



2/»o # 



2/i 



KA 



(5 Sch,) (5 Sch,) (5 Sch,) (5 Sch,) 
All All All All 



*SC Dist, Schl, Clara, 

tH-1) (K-5) (tf 10? 



KSC Diet. Schl. Clsrm, 
(»-3) (K-17) (K-34? 



BSC BSC BSC ha BSC; BSC; BSC; 

Dir. of Dir. of Dir., of 
Inst, Inst* Inst, 



I/sw. i/mo. 



1/fl 



HA 



(2 Sck.) 
All HA 



(2 Sch,) <2 Sch.) 
All All 



1-3 



l~o 



All 



All 



All 



All 



HA 



1-2 



1-4 



1-4 



T(i Sch.) 
C(2 Sch.) 



T(l $ch.) T(l Sch.) 
C(2 Sch.) C{2 SchJ 



l/i 



1/t 



l/*>. 



All 



All 



All 



17 



17 



17 



2-6 



All All 



KA 



KA 



All 



17 



2-6 



All 



All 
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Table B-l (cont,) 
Pennsylvania School District 



Frogrs* Element 


BSC 


Dist. 
(H-7) 


1981-42 
Schl. 


Clara. 
01-46) 


, Tear and Level of Response 

1982-83 

BSC 01st. Schl. Clsn. 

OH1) <*-5> <H-10> 


BSC 


Dist* 
(H-3) 


Schl. 
(H-17) 


Cisn* 
(IN34) 


Dis tr ict/Prlnc ipal 
Confer race s 

• Husber of parti- 
paring principals 


5 


5 

; 


5 


HA 


Hone HA Hone HA 


17 


17 


17 


HA 


• Participating cen- 
tral office staff 


1 


1 


1 


KA 




2 


2 


2 


HA 


• Mean nuasber of 
cemfs. per prinipal 


1 


1 


1 


HA 




1 


1 


1 


KA 


• Range in nuaber 
of conferences 


MA 




- HA 


• Variables discussed 

1 . 


All 


Ail 


All 


HA 




All 


All 


All 


HA 
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Table 0-2 



try of Program I«pl#»ent*tlon, 1981-84 
Hov Jer»*y School District 



Cfrll«ctlaiTof 

• VtrUblM *ttra4*4 
to CT. C, PL, AF) 




• for which «xbj«ct 
utu 



Of Cliff- 

lowlv»d 



# Participating 



• H^ma nMlir of rounds 
of obotrratlana 



of oooorrotlono 




Ail 



All 



All 



RA C 



HA 



Frlos; Rrin»; HA 
DOI mm DOI 



HA 



TtC.FL All 



All 



All 



T.C.H, All 



*U All All 

<••!-«) (g.l-S) 




^Tbo *up«rUt#nd#ac, 

4lDtmlm " indivldu.! ob-rvacion. vlch round, of obaarvacioo.. \ 
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BCST COPY AVAILABLE 

I 4 
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Table B-2 (coat.) 
*ev Jersey School District 





i — 










Tear and 


Lftvel of Be spans* 












1981-82 


19S2-*3 


ittt-aa 


Program Element 


BSC 


Diet. 
«-!> 


Schl. 
01-9) 


Clara. 


BSC 


Diet. 
Of!) 


Schl. 
/(N-8> 


Clam. 

fa=m 


BSC 


Diet/ 

<»-*> 


Seal. 


Clara. 
QMS) 


Principal leadership 
Seminars 














/ 












# Sealnar leader (s) 


BSC; 

Snpt; 

Aaat. 

Supt • 


Supt; 
Aaat. 
S«pt; 
Asst. 
to 

Supt; 
BSC 


MA 


MA 


Sopt; 
Asst. 
Sopt; 


Stept; / 
Asst. / 
Supt; 
Aaat* 

S«pt; 
BSC 


NA 


HA 


supt; 
Aaat* 
Supt; 
BSC 


Supt; 
Aaat. 

Supt; 
Aaat. 
to 

Sopt; 
BSC 


HA 


HA 


• Seminar frequency 


I /mo 


KA 


1/ao 


HA 


I/WJD 




I /mo 


HA 


1/ao 


1/ao 


1/ao 


HA 


rr inc l pea / Teac ne r 
Conferences 


























• Hwfcer of partici- 
pating teachers 


All 
<g.K-S) 


All 


All 

<g.t-«) 


HA 


All 

<g.i-S> 


All 

■VI A- 


At 1 

AAA 

<g.l-8) 




All 

All 

<g-i-S) 


Alt 

all 

<g.i-s> 


All 

(g-I-«) 


HA 


e Participating 
supervisors 


Prins; 
DOI* 


Prins. 


Prins; 

eooe 

001 


NA 


Prins; 
DOI 


Prins. 


Prins, 


Prins; 
Aaat, 
Prins; 
001 


Prins; 
001 


Prins. 


HA 


Prins; 
Aaat. 
Prins; 
001 


e Mean masker of confs. 
per teacher 


3 


3 


3 


4 


3 


3 


3 


4 


3 


3 


3 


2 


# Range in nuaber 
of conferencea 


2-4 


3 


2-3 


0-10 


2-4 


NA 


3-6 


2-8 




3 


3 


1-3 


a Variables discussed 


T,C 


All 


All 


All 


T,C 


All 


All 


All 


T 


All 


All 


All 



e D0I - Department of Instruction Staff (curriculum specialists). 



Table B~2 (coot.) 



New Jersey School District 



Prog r am Element 


BSC 


Diat. 
01-1) 


1981-412 
Sehl, 
<M-9> 


Clara* 
CN-1I6J 


BSC 


Tear and Level of Response 
1982-83 
Dist* ScM* dsns. 
iM^l (1-18) 


BSC 


Dist, 
<»-*> 


1983*84 
Scbl* 
OH8) 


Clara. 
(K-18) 


District /Principal 
Conferences 

0 Nuaber of partici- 
pating principals 


9 


9 


9 


MA 


a 


8 


8 


RA 


8 


8 


8 


* 

MA 


e Participating cen- 
tral office staff 


Supt* 


Supt. 


Supt. 


HA 


Snpt. 


Supt* 


Supt. 


MA 


* 

Supt . 


Supt* 


Supt* 


NA 


4> ffean of confs* 
per . 1 


2 


2 


NA 


NA 


2 


2 


2 


NA 


2 


2 


i f 


MA 


e Range in masher 
of conferences 


2-3 


2 


RA 


MA 


2 


2-4 


1-4 


MA 


1-3 


2 


1-3 




0 Variables 
f Discussed 


T,C,PI. 


All 


All 


NA « 


T,C,AP 


/ll 


All 


NA 


-.c 


All 


All 


MA 



of date of interviews one more conference was planned* 



178 



179 



Table B-3 



Sumnary of Program Implementation, 19B1-S4 
Delaware School District 















Tear mat 


1 Lewi of Besmoase 












1961-32 


1982-83 


1983-84 C 


Program Element 




Dist. 


Schl.. 


Clsrm. 
CH-5) 


BSC 


Dist. 


Schl. 
0M) 


Clsrm. 
OM) 


BSC 


Dist. 
0W> 


Schl* 
(H-D 


Clsrm* 
00-3) 


Collection of Clais* 
room Data 

e Variables attended 
to (T f C,PL,AP) a 


All 


All 


All 


T,PL, 

C 


T,PL 


T.PL 


T*PL 


T t PL 


PL 


none 


PL 


PL 


e For which subject 
areas 


8/ LA 
if 

mm 


HA* 


HA 


HA 


H/ LA 
H 


HA 


HA 


HA 


R/LA 
M 




HA 


MA 


e Komber of class- 
rooms involved 


All 


All 


HA 


HA 


1 Sch. 


1 Sch. 


HA 


HA 


HA 




HA 


■A 


e Participating 
Supervisors 


2 Prin*. 
1 Asst. 
Prin. 


2 Prins. 
1 Asst. 

Prin. 


1 Prin. 
1 Asst. 
Prin. 


1 Prin. 
1 Asst. 
Prin. 


1 Prin. 


1 Prin. 1 Prin. 


1 Prin. 


Dir. 
of 

Spec. 
Pgms. 




1 Prin. 


t 


s Mean number of 
rounds of 
observations 


2 


HA 


HA 


HA 


2 


HA 


1 


1 


none none none none 


e Range of observa- 
tions 


1-4 


HA 


HA 


ft* 


1-2 


MA 


0-2 


0-2 




tT 







£rhe superintendent and the director of special programs. 
c "lVo principals and one assistant principal. 

^963-84 principal and teacher interviews were from one elementary school; the principal of the other elementary school left the district 6/83, 
^T - Time; C - Content; PL • Prior Learning; AP - Academic Performance. 
HA - Hot asked of interviewee. 



Hots; Data sources Include BSC observations and Contact Reports; superintendent Interviews; principal and teacher interviews and questionnaires 
principal/ teecber conference '*>~ms. 



Table B-3 (cont.) 
Delaware School District 

t 



Program Kleseat 




1981-82 






Tear and Level of 
1982-83 


Response 




BSC 


Dlst. 


Schl. 
(N-3) 


Clara. 
(N~5> 


BSC 


Dist. 
<*-2) 


Schl. 
<N-1> 


Clsra. 


BSC 


Dist. 
(N-2) 


Schl. 
(N-l) 


Clsrs. 
(N-3) 


Principal Leadership 
Seminars 


























m Seminar leader(s) 


BSC; 

Supt; 

Dist. 

Staff; 

Con- 
sul- 
tants 


BSC; 

Supt. ; 

Dist. 

Staff; 

Con- 
sul- 
tants 


BSC; 

Supt. ; 

Dlst. 

Staff; 

Con- 
sul- 
tants 




Supt; 

Dlst. 

Staff; 

Con- 
sul- 
tants 


Supt; 

Dist. 

Staff; 

Con- 
sul- 
tants 


HA 


NA 






* 


NA 


# Seminar frequency 


4/yr 


HA 


NA 


NA 


i-2/mx, 


I-2/mo 


NA 


NA 


none 


none 


none 


NA 


Pr inc i pa 1 /Teache r 1 
Conferences 


























• Number of partici- 
pating teachers 


All 


All 


HA 


NA 


1 sch. f 


1 sch.* 


none 


NA 


none 


none 


none 


NA 


0 Participating 
supervisors 


1 Prin. 
1 Asst. 
Prin. 


JIA 


1 Prin* 
1 Asst. 
Prio. 


1 Prin. 
1 Asst. 
Prin. 


1 Prin. 


1 Prin. 


none 


none 


none 


none 


none 


nose 


• Mean number of 
confs. per teacher 


2 


NA 


NA 


3 (Tchrs. 
paired) 


1 


NA 














# Range in nujeber 
of conferences 


HA 


NA 


HA 


NA 


0-2 


NA 








• Variables discussed 


All-1 

Sch 
T - 1 

Sch 


HA 


NA 


T 


T 


• 

HA 




," 





Conference using engagement rate form, net principal/teacher conference form. 
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Table 8-3 (cone) 
Delaware School District 





r— — , . Tear and Level of Reenon** 


Program Elenent 

District/Principal 


1981 —if 7 

BSC Dtst. Schl, Clara. 

(N-2) 01-3) (||-5) 


1982-83 

BSC Dist. Schl. Clara. 
(H-2) (W-1) (m.3) 


WC Dlst , Schl M Clan. 

<*-2) tN-1) (IM) 


Conferences 

• Nunber of partici- 
pating principals 


1 Prin. 
1 Asst. 

Prin. 2 1 ha 


none none none NA 


none none none NA 


• Participating cen- 
tral office staff 


Dir. Dir. Dir. 
of Sp. of Sp. of Sp, 
PgM. Pgias. PgM. NA 


NA 


NA 


• Mean nuaber of confs. 
per principal 


15 RA «A NA 


- NA 


NA 


• Range In number 
of conferences 


1-2 HA HA NA 


- - NA 


NA 


• Variables Discussed 


All All All NA 


- NA 


NA 
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Appendix C 
Significance of Achievement Cains 
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ERIC 



SIGNIFICANCE OP ACHIEVEMENT GAINS 

According to the Norm-referenced Title 1 Evaluation Model (model A), 
the treatment group is assigned to increase in achievement at the same rate 
as the norm group. Thus, an NCE change of zero indicates the group 
progressed at the same rate as a representative sample of students at that 
grade level. NCE changes greater than and less than zero can be attributed 
to the intervention. 

How much change is necessary to be deemed educationally significant? 
This issue is fraught with ambiguity — no easy rule of thumb has been agreed 
upon* Some researchers suggest that a change of one-third of a standard 
deviation is significant. In this case, seven NCEs would be appropriate 
(the standard deviation for NCEs is 21.06). Other researchers ind/cate 
that any change (i.e., one NCE) is good (Tallmadge & Wood, 1976; Tallmadge, 
1976). BSC chose here simply to present the data and point out apparent 
trends, rather than base interpretations on an unconfirmed standard. 
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